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The  Invariance  Principle  for 

Depende nt  Random  Variables 

By 

Patrick  Paul  Billingsley 

(Abstract) 

In  this  paper  the  Erdos-Kac  invariance  principle,   as 
generalized  by  Donsker  (Mem.   Am.   Math.   Soc .  ,    no.  6  (1951),) 
is  extended  to  the  dependent  case.     Let    C    be  the  space  of 
functions  continuous  on  the  closed  unit  interval,    with  the  uni- 
form topology.     If    {  X     }     is  a  sequence  of  random  variables 
on  a  probability  space     ( -O. ,    W^  ,  P),     let    p       be  that  elemient 
of    C    which  is  linear  on  each  of  the  intervals     ( ( j  -  1 )  n"  ,  j  n      ) 

and  satisfies    p    (0)  =  0    and    p    (jn*  )  =  X,  +  •  •  •   +  X.     - 

^n  ^n -^        '  1  J    for 

j  =  1 n.     Thus    p       is  a  (measurable)  mapping  of  JDL    into    C  . 

Suppose  there  exists  a  sequence     (a     }    of  positive  constants 

such  that  if  the  measure    P       is  defined  by    P    (A)  =  P{a"    p      G   A} 

n  '        n  n     *  n 

for  measurable  subsets    A    of    C  ,    then    {P    }     converges  weakly 
to  Wiener  measure.     If  this  is  true  we  say  that  the  invariance 
principle  holds  for  {  X     }   .     Donsker  has  shown  that  the  in- 
variance principle  holds  if  {  X     }    is  independent  and  stationary 
and    X-     has  zero  mean  and  unit  variance.     In  the  present  paper 
we  prove,    after  some  measure-theoretic  prelinninaries,    that  the 


invariance  principle  holds  under  each  of  the  following  condi- 
tions,     (i)    X     =  f(x    ),    where    f    is  a  function  on  the  state  space 
n  n 

of  a  discrete  Markov  process    {x    }    satisfying  Doeblin's  con- 
dition,    (ii)    {  X     }    is  m-dependent.      (iii)    {  X     }    is  a  discrete 
n  n 

linear  process  with  m-dependent  residuals,     (iv)    X       is    1    or    0 
according  as  a  recurrent  event  occurs  or  not  at  the    nth    of  a 
sequence  of  trials.     In  each  of  these  four  cases  the  additional 
assumptions  under  which  the  invariance  principle  is  proved 
are  essentially  those  under  which  the  corresponding  central 
limit  theorenn  has  been  proved. 
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So.     Introduction. 

In   [9]*    Erdos  and  Kac  introduced  a  new  method  for  proving 
weak  limit  theorenas  for  functions  of  the  partial  sums  of  an  inde- 
pendent sequence  of  randonn  variables.     Their  nriethod  consisted 
in  showing  first  that  the  limiting  distribution  is  independent  of 
the  particular  sequence  and  then  conaputing  this  distribution  for 
some  convenient  sequence.     Let 

(0.1)  X^,    X^,    ... 

be  an  independent  sequence  of  identically  distributed  random 
variables  with  zero  means  and  unit  variances.     Let 
S,    =  X,  +  •  •  •   +  X,    .     Erd'os  and  Kac  showed  that  the  limiting  dis- 
tribution of 

(0.2)  n'^/^      max      S, 

(along  with  several  other  functions  of    S, ,    ....    S^)    is  independent 
of  the  distribution  function  conamon  to  the  variables  in  the  sequence 
{Xj^}  .     Since  the  limiting  distribution  of    (0.2)   in  the  Bernoulli 
case  was  well  known,   this  argument  gave  the  limiting  distribution 


*Numibers  in  brackets  refer  to  the  bibliography  at  the  end  of  the 
paper.     The  expression  "Theorem  i.j"    refers  to  the  jth  theorena 
of    §i  ,   while  "  Theorem  A.  j"    refers  to  the  jth  theorem  of  the  ap- 
pendix. 
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of   (0,Z)   under  quite  general  conditions.     There  followed 
several  papers    (cf .  ,    e.g.,    Erdos  and  Kac   [10]  and  Mark  [17]) 
in  which  this  argument,   known  as  the  invariance  principle,    was 
applied  to  various  other  functions  of  the  partial  sums    S^  .     All 
of  these  results  were  subsequently  subsumed  under  a  general 
theorem  due  to  Donsker    [6]  . 

Donsker's  result  runs  essentially  as  follows.     Let   C   be 
the  space  of  functions  x  (t)   continuous  on  the  closed  unit  interval, 
with  the  unifornn  topology,   and  let    W   be  Wiener  measure  on   C  . 
Let    (Xn)   be  a  sequence  of  randonn  variables  on  some  probability 
measure  space     {S^,  CB  •   P  )  •     Let  pj^  be  that  element  of  C   which 
is  linear  on  each  of  the  intervals     (  (j  -  l)n"  ,   jn"    )  ,    j  =  1,    .  .  .  ,    n, 
and  satisfies  Pj^(jn"  )  =  S-   for  j  =  l,    ...,   n,    and  Pn(0)  =  0.     That 
is,   let  p^  be  the  random  function  defined  by 

(0.3)  Pn(t)  =  S.^j+(nt-j  +  l)Xj,  (j-l)n*^<t<jn"\    j  =  l n, 

where    Sq  =    0  .     Thus  p^^   is  a  (measurable)  mapping  of  S\  into   C  . 
Let  f  be  any  function  on   C   which  is  continuous  in  the  uniformi  top- 
ology at  almost  all  (W -measure)  points  of  C  .     Donsker  showed 
that  if    (0.1)   is  an  independent  sequence  of  random  variables  which 
are  identically  distributed  with  zero  mean  and  unit  variance,   then 
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lim      P{f(n"^/^p^)<a}    =     W{x:  f(x)<a} 


T>-*«0 


at  continuity  points    a   of  the  function    W{x:  f(x)<a}  .     If,    for 
example,    f(x)  =  max     x(t)  ,    then  f    satisfies  the  above  conditions 
and   f  (n"  '     p    )  =  n'  '       max   Si.  ,     so  that 

{0  ifa<0 

a 
^_^e-"'/^du    ifa>0, 

where  the  right  hand  equality  can  be  established  by  any  one  of  a 
number  of  methods.     See    [6]  for  other  functions  f  which  lead  to 
interesting  linnit  theorenns. 

It  should  be  pointed  out  that  in  place  of  the  "  random  polygon" 
p^  defined  by    (0.3)  ,   Donsker  actually  worked  with  the  "random 
step-function"   with  value   S:    throughout  the  interval 
((j  -l)n"  ,    jn"  ]  .     There  is  of  course  no  essential  difference 
between  the  two  methods. 

There  is  another  way  of  stating  Donsker 's  result.     Let    C 
be  the  Borel  field  generated  by  the  open  (uniform  topology)  sub- 
sets of  C  .     Suppose  there  exists  a  sequence    {aj^}   of  positive 
constants  such  that  if  Pj^  is  a  measure  defined  by  setting 
Pn(A)  =  P{aj[^     Pjj  ^A}    for    A  €  C  ,    then    Pj^  converges  weakly 
to    W  .      When  this  is  true  we  say  that  the  invariance  principle 
holds  for  the  sequence    {Xjj}    with  norming  factors    a     .     Then 
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(cf.    Theorem  1.1  below)   Donsker's  result  says  that  the  invariance 
principle  holds,    with  norming  factors    n  '      ,    provided    {Xj^}    is  an 
independent  stationary  sequence  with    E{X,}   -  0   and    E|X.}  =  1. 
The  assumption  that    {X^^}   is  stationary  is  relatively  unimportant. 
It  is  the  purpose  of  the  present  paper  to  replace  the  assumption  of 
independence  by  various  weaker  hypotheses. 

The  plan  of  the  paper  is  as  follows.     The  central  results  are 
contained  in    §§4   through    7,   those  of    §§1  through    3   being  prelim, 
inary.     These  first  three  sections  are  devoted  to  an  account  of  the 
theory  of  weak  convergence  of  probability  nneasures  on     vS    (  §  1  )  . 
an  alternative  proof  of  the  existence  of  Wiener  measure    (  §  2  )  and 
a  general  invariance  principle    (  §  3  )  . 

In    §  3   we  sort  out  those  steps  in  the  proof  of  Donsker's 
theorem   (Theorem  1   of   [6]  )  which  depend  upon  the  assumption 
of  independence  and  state  them  as  the  hypotheses  of  Theorem   3.1. 
This  theorem  then  gives  a  set  of  conditions  on  the  sequence    {Xj^} 
which  insures  that  the  invariance  principle  holds  with  a  suitable 
sequence  of  norming  factors.     While  these  conditions  are  not  very 
pleasing,   they  can  be  verified  for  those  sequences    {X^}   of  great- 
est interest. 

Theorems    1.1  and  1.3   are  preliminary  to    §3.     Theorem  1.1 
gives  several  sets  of  conditions  equivalent  to  weak  convergence. 
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These  conditions,    with  the  possible  exception  of    (ii),    are  well 
known.     Theorenm   1.3,   on  which  Theorem  3.1  depends,   gives 
a  simple  criterion  for  weak  convergence  of  probability  measures 
on    C    in  terms  of  the  convergence  of  the  measures  of  sets  of 
the  form 

{  x:    ttj  <  x(t)  <  pj  ,    (j  -  l)c"^  <  t  <  jc'^  ,    j  =  1,    c}  . 

where   c   is  a  positive  integer  and  a-  ,    p.   arbitrary  real  nunnbers. 
This  theorem  is  a  slight  generalization  of  one  due  to  Donsker. 
The  proof  differs  from  his  in  that  several  arguments  "of  the 
Riemann  approximation  type"   are  eliminated,    which  elimination 
is  miade  possible  by  condition    (ii)   of  Theorem  1.1.     Theorem  1.4, 
which  is  essential  to  the  considerations  of    §4  ,    is  the  anologue  for 
distributions  on   C    of  a  well-known  limit  theorem  for  distributions 
on  the  real  line.     Its  proof  depends  on  condition    (ii)   of  Theorem  1.1  . 

Theorem   1.2   and    §2   are  a  side  issue.     Theorem  1.2   is  a 
result,    announced  by  Prohorov   [19]  ,    on  the  weak  compactness  of 
measures  on    G    .     Prohorov  has  used  this  theorem  to  give  an 
elegant  proof  of  the  invariance  principle  in  the  independent  case, 
but  his  method  seems  difficult  to  apply  in  those  cases  to  which 
the  present  paper  is  devoted.     His  result  is  really  an  existence 
theorem,   and  in    §3    we  use  it  to  prove  the  existence  of  Wiener 
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measure.     This  method  of  proving  the  existence  of  a  stochastic 
process  is  of  course  not  very  general    (cf.  [8,    Ch.   II]   for  a  gen- 
eral approach)   but  an  alternative  proof  of  this  important  theorem 
is  interesting. 

In    §§4   through    7  ,    Theorem  3.1   is  specialized  in  various 
ways.     In    §4  the  invariance  principle  is  proved  for  sequences 
{f(Xjj)}  ,    where   f   is  a  function  defined  on  the  state  space  of  a 
discrete  Markov  process    {xj^}    satisfying  Doeblin's  hypothesis. 
The  conditions  under  which  this  result  is  obtained  are  identical 
with  those  under  which  the  central  limit  theorem  for  such  processes 
is  proved  in   [8]  . 

In    §5   we  prove  the  invariance  principle  for  nn-dependent 
sequences  of  random  variables.     The  best  central  limit  theorenms 
for  such  sequences  are  due  to  Marsaglia   [18]  ,    and  the  conditions 
under  which  the  theorems  of    §5    are  proved  are  essentially  those 
of  his  central  limit  theorems.     Donsker's  original  theorem  follows 
from  the  results  of    §5  . 

§6  treats  of  discrete  linear  processes  with  m-dependent 
residuals,   processes  which  arise  in  the  analysis  of  time  series. 
Here  we  prove  the  invariance  principle  under  conditions  only 
slightly  stronger  than  those  assumied  by  Diananda   [5]   in  his  proof 
of  the  central  limit  theorem  for  such  processes. 
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Finally,  in  §7  we  prove  the  invariance  principle  for  the 
number  of  occurences  of  a  recurrent  event.  Here  we  assume 
that  the  recurrence  time  has  a  finite  second  moment. 

In  the  appendix  we  prove  several  limit  theorenns  for  c- 
dimensional  distribution  functions.     These  theorems  are  all 
routine  extensions  of  results  well  known  for  the  case    c  =  1  . 

It  is  doubtful  that  the  results  of    §§4    through   7    can  be 
substantially  improved  using  present  nnethods,    since  in  each 
case  the  invariance  principle  is  proved  under  conditions  virtually 
the  same  as  those  under  which  the  central  limit  theorem  has  been 
proved.     It  is  possible  to  prove  the  invariance  principle  in  cases 
other  than  those  considered  here.     One  can,   for  example,   prove 
it  for  nnartingales,    as  Levy   [16]   has  the  central  limit  theorem, 
or  under  the  assuiTiptions  of  Bernstein's  "  lemme  fondannental" 
[2]  .     Although  no  applications  have  been  essayed,   the  cases 
treated  in    §§4   through   7    are  those  of  greatest  interest  for  the 
applications. 
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§1,     Convergence  of  measures  on  the  space  of  continuous  functions. 

In  this  section  we  prove  two  useful  theorems  on  the  convergence 
of  probability  measures  on  the  space  of  continuous  functions. 

Consider  first  an  arbitrary  nnetric  space   56    with  metric    P    . 
In  what  follows  we  will  be  interested  in  the  cases  in  which    9S     is 
either  the  space  of  continuous  functions  or  a  Euclidean  space.     Let 
(i3  be  the  collection  of  Borel  sets,    that  is,    the  Borel  field  gener- 
ated by  the  open  sets.     If    Pj^,    P    are  probability  measures  on    vj)  , 
we  say  that     P^^    converges  weakly  to     P    (in  symbols,     P^  r"^  P)   if 


\        f    dPn      ^^     \        f    dP 


3e  3g 

for  all  bounded  continuous  functions     f  . 

Theorem  1.1  gives  several  convenient  sets  of  conditions 
equivalent  to  weak  convergence.  For  its  proof  we  require  the 
following  variation  on  Urysohn's  lemnna, 

Lemma    1. 1  .       If    A    and    B    are  closed  sets  with       /^(A,  B)>  0, 
then  there  exists  a  function      f(x)    which  is     1    on    A,     0    on    B  , 
everywhere  between    0    and    1  ,   and  uniformly  continuous  on  3€    . 

Proof;        We  may  of  course  assume  that    A    and    B    are  non-ennpty. 
V/ith  the  exception  of  unifornn  continuity,   it  is  clear  that  the  function 
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f(x)    = 


^(x.  B) 


^(x.  B)  +^(x.  A) 


has  the  required  properties.     To  prove  that    f    is  uniformly 
continuous  observe  first  that     (cf,    [l,   p.  57]) 


\  p  (x,  A)  -My,  A)\     <    /9  (x,  y) 


and 


a  (x.  A)  +    a  (x,  B)  >   /P  {A,  B)  . 

From  these  inequalities  it  follows  that 

|f(x).f(y)|     <l/^(x.B)-/^(y.B)| 
y?(x,  B)  +yQ(x,  A) 

+  /?(y.  B) 


^(x.  B)  +  /i>(x,  A)  ^(y,  B)  +^(y.  A) 


<    r\^'  l\    +  /^(y>^)  |/:>(y.B)-/^(x.B)|+|/P(y,A)-^(x,A)| 

-    /5(A,B)  /9(y.B)  +  /9(y.A)  /?  (x.  B)  +  /?  (x.  A) 


/^' 


^(y.B)  +  /? 


/7(x.  B)  +  /? 


/^ 


(A.  B) 


r 


(x,  y)  . 


Hence     f    is  uniformly  continuous   on     3^    . 

In  what  follows  we  denote  the  closure,    interior  and  boundary 
of  a  set    A    by     A,    A    and    A,    respectively.     If    P    is  a  probability 
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measure  on    36     and    f    is  a  nneasurable  function  then     P{x:  f(x)<a} 
is  a  function  of    a    which  we  call  the  P-distribution  function  of    f  . 

Theorem  1. 1 .       The  following  statennents  are  equivalent, 
(i)  Pn=>P  • 

(ii)        \        f    dPj^   — ^-  \        f    dP    for  bounded  uniformly  continuous 

functions    f  . 

(iii)      P(A)>lim    sup      Pn('^)    fo^  closed  sets     A  , 

(iv)       P(A)  =  lim      Pn(A)    for  sets     A  ^  (8   such  that    P(A)  =  0  . 

(v)        For  any  function    f    which  is  continuous  except  on  a  set  of 

P-measure  zero,    the    Pj^-distribution  function  of    f    converges 
to  the  P-distribution  function  of    f    at  each  continuity  point  of 
the  latter. 

Proof.       We  will  prove  in  turn  the  implications     (i)— *•  (ii )  — ^(iii)  — *■ 
(iv)— ^(v) — *(i)  .     The  implication    (i) — *^(ii)    is  trivial. 

(ii)  — *-(iii)  .       Suppose  that    A    is  closed  and     C  >  0    given.      We 
may  assume  that    A    is  neither  the  empty  set  nor  the  whole  space. 
For     S    >  0    let    U        =   {x:    /3  (x,  A)  <    S  }  .     Then    U^     is  open 
and    U-  ^  A    as     S  |   0  ,    since     A    is  closed.     Hence  there  exists 
a      S     such  that    P(U^    -  A)  <   £  .     Clearly     /o(A,9e-U^)>5>0. 
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Therefore,    by  Lemma  1,1,    there  exists  a  uniformly  continuous 
function    f    which  is     1    on    A  ,     0    on  36  -  U-     and  everywhere 
between     0    and    1 .     Now 

f       f    dP^    -^  (j^       f    dP 

by     (ii )  ,   and 


J 


f    dP„>P„(A). 


while 
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r     f    dP  <  P(A)+  P(U^- A)  <  P(A)+    e    . 
From  these  three  relations  it  follows  that 


lim  sup  P„(A)    <    P(A)+    6    . 
n  — ^  oo 

Since     £     is  arbitrary,      (iii)     follows. 

(iii)-^(iv).       Suppose  that     P{A)  =  0  .      Then 

(1.  1)  P(A)  =  IMA)  >  lim  sap  P„{A)  >  lim  sup  P_(A)  . 

n  — *-  «»o  "  n  — *-  oo 

Since  the  boundary  of    9C  -  A    also  has    P-measure  zero  we  have  in 
the  same  way. 


(1.2)  P(Be -A)  >  lim  sup  Pr,(3€  -  A) 

h  — >■  oo 

But     (1.1)   and    (1.2)   imply 


P(A)  =   lim     Pn(A)  . 


(iv)  — »-(v).       Let     F       and     F     be  respectively  the    P^^ -distribution 
functions  and  the    P-dis tribution  function  of    f    and  let     A     be  the 
set  of  points  at  which     f    is  discontinuous.      Then 

{  x:    f(x)  <  a}      C    {  x:  f(x)  <  a}  w  A 
and 

{x:  f(x)  <  a}     -    A      C   (x:   f{x)  <  a}°    , 
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so  that  the  boundary  of     {x:  f(x)  <  a}    is  contained  in     {  x;    f (x)  =  a)  v-»    A  . 

Since    P(A)  =  0  ,     F(a)  =  F(a-0)    implies  that  the  boundary  of 

{  x:  f(x)  <  q}    has    P-measure  xero,  and  hence  that    .F„{a) -»- F(a)  . 

(v)  — »-(i)«       Let    F      and    F    be  the    P^-distribution  functions  and 
the  P-distribution  function  of  the  bounded  continuous  function    f  . 
We  assunne  that    F^(a)-»'F(a)    if    F    is  continuous  at    a    and  must 
show  that 

\      f    dP„  ^         ^     f    dP  . 

But  this  last  statement  is  equivalent  to 

\  a    dF    (a)       3-     \  a    dF(a) , 

where    M    is  the  bound  of    f  .     But   (1.3)  is  easy  to  establish    (cf. 
[3,  p.    74]).     This  completes  the  proof  of  Theorenn  1.1. 

We  note  at  this  point  the  well-known  fact  that  if    36     is 
c -dimensional  Euclidean  space  then  equivalent  to  each  of  the  con- 
ditions of  Theorem  1.1   is  the  condition  that  if    F       and    F    are  the 

n 

distribution  functions   corresponding  to     P       and     P    respectively, 
then 


lim         ^n^°r    •  •  •  •    *^c^  ~  ^^*^i'    •  •  •  '    °c^ 


at  each  point    (a  ,    , .  .  ,   a^)    surh  that 
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F(aj^,    ....    a,,)  =  sup      F(p^ p^)  , 

i.e.  ,   at  continuity  points  of    F  . 

Let    C    be  the  space  of  functions    x(t)    continuous  on  the 
closed  unit  interval,    with  the  metric 

/9(x.  y)   =    sup      |x(t)-y(t)|    . 
'  o<t<l 

Then    C    is  a  connplete,    separable  metric  space.     Let    C     be  the 

collection  of  Borel  sets.     The  Borel  field     C    is  generated  by  the 

sets  of  the  form     {x:  x(t)  <  a}  .     That  such  sets  belong  to     C     is 

obvious,   and  to  see  that  they  generate     yZ    it  is  enough  to  observe 

that 

{x:  ^(x,Xo)<    8    }    =    n    (x:   |x(r)-Xo(r)|   <    S     }  . 

where  the  intersection  extends  over  all  rationals    r    in  the  unit 
interval. 

If    t,,    ..  .,   tj^    are  fixed  points  in  the  closed  unit  interval, 
[x(tj),    ...,  x(tj^)]    defines,   as    x    varies  over    C,  a  k-dimensional 
random  vector  on    C    which  we  denote     [x     ,    ...»  x     ]  , 

The  first  of  the  two  theorems  concerning  the  convergence  of 
probability  measures  on  C  which  we  will  need  is  due  to  Prohorov 
[19]  . 
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Theorem  1.  2  .      Suppose  that     l^n^    ^^  ^  sequence  of  probability 
measures  on     O   with  the  property  that  for  each    €   >  0    there 
exists  a  compact  set    K         such  that    Pj^(K  £  )  >  1-  £     for  all    n  , 
Then  there  exists  a  sequence     {n      }    and  a  probability  measure 
P    such  that    P       :=f^  P    as    V— »-oo  , 

Proof.       Let    {tj^.   k  >  1}    be  an  ordering  of  the  rationals  of  the 
closed  unit  interval.     For  each    n    and    k 

(1.4)  y^   ^  (S)   =   P^(&c^  .....  x^    ]  €S} 

*  Ik 

is  a  probability  measure  of  k*dinnensional  Borel  sets    S,     Let 

F,  be  the  distribution  function  corresponding  to     P  ,         ,     For 

K  ^  n  I   ic  f  n 

each    k    it  is  possible  by  Helly's  theorenn  to  find  an  increasing 
sequence     (n^  }    of  integers  and  a  function    F,(a.,    ....   a    )    such 
that    F^    is  everywhere  between    0    and    1  ,     is  non-decreasing  in 
each  variable,   is  continuous  from  above  and 

(1.5)  lim        F^  ^     (a^ a^)   =    Fj^(aj.    ...#    \) 

at  continuity  points  of    F\^  .     By  the  diagonal  method  it  is  possible 
to  choose  a  single  sequence     {n^  }    so  that   (1.5)  holds  for  all    k 
simultaneously. 

For  a  fixed    k    let 


Sg    =  {[x(r^).    ....  x(r^)]:      x  6  K^  } 
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Then    S  ^    is  a  k-dimensional  Borel  set  with 
£ 


Vk.n'^e* 


>  1   -    £ 


for  all    n  ,     Since    Kg    is  compact,    S  g    is  bounded.     Frora  these 
facts  it  follows  that    F,      must  have  total  variation    1,    i.e.,    that  it 
is  the  distribution  function  corresponding  to  some  probability 
measure      y,    .     And  from  the  remark  following  the  proof  of 
Theorem  1. 1  we  conclude  that 

(1.6)  p^^^     :r>      fk  (v>-^«>     ) 

for  all    k  . 

We  now  use  the  measures     P,      to  set  up  a  measure  on    C 
in  a  way  similar  to  that  used  by  Kolmogorov    [15]   in  his  fundamental 
existence  theorem.     His  theorem  as  such  is  not  applicable  here  since 
we  are  working  in  the  space  of  continuous  functions  rather  than  the 
space  of  all  functions.     Let     t     be  the   (finitely  additive)  field  of  sets 
of  the  form 

(1.7)  A    =    {x:  [x(rj) y.{r^)]    e  S}  , 

where    k     is  any  integer  and    S    is  a    k-dimensional  Borel  set.     For 
such  a  set    A    put     P(A)  =     Pu(S)  .     Since  there  are  other  represen- 
tations of    A  ,   we  must  show  that  this  definition  is  consistent. 
Suppose  then  that  in  addition  to    (1.7)   we  have 
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(1.8)  A   =    {x:  [x(r^) x(r.)]    G  S'}  . 

where    j  >  k    and    S'    is  a    i-dimensional  Borel  set.     From  the  fact 
that  for  any  point     (  ^,,    •  •  •  »    *»  u)    ^^   k-space  there  exists  an' 

X  e  C    with    [x(r. ),    .  .  . ,   x(r,  )]    =    {  ^^ ^  k^    ^*  follows  that 

S'   =    {(^1 ^   •):    (^, ^k^    ^    S}.     From  this  and 

the  definition    (1,4)  we  have 

for  all    n  .     Hence  by    (l.b) 

(1.9)  Pk<^^  =     Pj^^^l ^j^'<^l <i^)^S}. 

provided  the  boundary  of    S    has     V  .  -measure  zero.     But  this 
clearly  implies    (1.  9)   for  all    S  ,   which  establishes  the  consistency 
of  the  definition  of    P(A)  , 

It  is  easy  to  show  that    P    is  a  finitely  additive  measure  on 
^    and  that    P(C)  =  1  .     We  now  prove  that    P    is  completely  addi- 
tive on     T"    .     Suppose  then  that     {A,  }    is  a  non-increasing  sequence 
of     ^    sets  with    P(Aj^)  >  L  >  0    for  all    k  .     We  will  show  that  the 
Aj^    have  a  non-empty  intersection.     For  notational  convenience  we 
assume  that    A,      is  defined  in  terms  of  the  first    k    of  the     {r^}  : 

Aj^    =    {x:  fxCr^) '^^'^k^^     ^    ^k  ^     • 
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s 


Choose     S    so  that    0  <  2  6  <  L    and  let    Kg    be  as  in  the  hypothesi 
of  the  theorem.     Let    Uj^  ^  ^W    t)e  a  compact   k-dimensional  Borel 
set  such  that       Pk^^'^k^"^    62        .     Now  let 

V^   =    {x:   [x(r^),    ...,   x(r^)]      ^      U^}  .     let    W^   =    V^- -  •  V^    and 
let     Zj^   =         {[x(r,) x(r,  )]:xeW,  }  .     Then 

(1.10)  f^k<\"^k^   "    ^^^k"^'k^-    ^ 

Finally,   let    J,    =   ([x(r,) x(r,  )]  :    x  €Kg  }  ,     Now    J,      is  com- 
pact and  hence  closed.     In  fact,   if     ^      ^  ^^k    ^°^    '^  ^  1  »   o^ie  can 
select    Xj^  6  Kg       such  that    [^^^(r^),    ....  Xj^(rj^)]  =    ^^-     Since 
Kg      is  compact  there  exists  a  sequence     {m^}    of  integers  and  an 
X  €  Kp       such  that        linn    x        =  x  .     But  then 

lim         ^         =    [x(r,),    ...,   x(r,  )]  €  J,    ,    so  that    J,      contains  a  linnit 
point  of  the  sequence     {  ^ ^n^  •     Since    J^    is  closed  we  have  by    (1.6), 

(1.11)  U,  (J.)  >   lim    sup     U,  (J.)>limsupP        (K  ^  )  >  I  -  £ 
From    (1.10)   and    (1.11)   it  follows  that 

l-k'Vk'^   Kk'V-2^     >L-2   £    >0. 

Hence     Z,  J,      is  non-empty,   which  inaplies  that    W^i^K  ^    is  non-ennpty. 
Since  the     \\ ,      form  a  non-increasing  sequence  of  sets,   and  each  is 
obviously  closed,    the     ^^^K         have  a  non-empty  intersection  by  the 
compactness  of    K  ^      .     Since     V/      C   A      ,   the    A       have  a  non-empty 
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intersection.     Hence     P    is  connpletely  additive  on       -j-  • 

Since     -j'   generates     C  ,    P  can  be  extended  to  a  probability 
nneasure  on      C  .     We  now  complete  the  proof  of  the  theorem  by 
showing  that    P       =^    P  ,     By  Theorenn  1.1  it  suffices  to  prove  that 

(1.12)  P(A)  >    lim  sup      P       (A) 

for  closed  sets    A  .     For  a  given     £    >  0    let 

Sj^   =    {[x(r^),    ....   x(rj^)]:    x    G  AK  ^    }  . 

Now    S,     is  closed  for  the  same  reason  the  set    J,     above  was.     If 
Bj^    =    {x:  [x(r,),    . .  . ,   x(r,  )]    €  S,    }  ,   it  is  clear  that     {Bj^}    is  a  non- 
increasing  sequence  of  sets  of     C  with    AK     C  I  1    B,    .     Suppose  on 
the  other  hand  that    x  €  N,  B,    .     Fronn  the  definition  of    S,     and    B, 
it  is  possible  to  find  a  sequence     (ym)    °^  points  of    AK  g       such 
that    yjYi(r .)  =  x(r .)    for     i  =  1,    ....   k.     Since    AK  g      is  compact 
there  exists  a  sequence     {m^}    of  integers  and  a  point     y  €AKg 
with      linn    y        =  y  .     But  then    x  =  y     by  continuity,   and      x  ^  AK_     , 
Hence    AK^   =   H,  B,    .     It  is  therefore  possible  to  choose    k    so  that 

(1.13)  S+  P(AK^  )>  P(Bj^)  =     Pi^(Sj^). 

Since    S,     is  closed  we  have  by    (1.6)  and  Theorem  1.1  , 

(1.14)  y^iS    )>    Umsup     U  (S^)    . 
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Now  by  the  hypothesis  of  the  theorem, 

\  k.n^     k      -       n^  S  n^ 

for  all    v»     .     Hence 

(1.15)  lim  sup      U,  (S    )  >  lim  sup  P       (A)  -    £ 

But  now    (1.12)   follows  from    (1.13),    (1.14)  and    (1,15),    since     £      is 
arbitrary.     This  completes  the  proof  of  Theorem  1.2. 

In  order  to  apply  this  theorem  we  need  a  criterion  for  coin- 
pactness  of  sets  in    C  .      The  following  standard  lemma  gives  a 
convenient  criterion  in  terms  of  the  modulus  of  continuity,   which 
for  our  purposes  is  best  defined  by 

M(x.  S  )  =  sup  {|x(s)-x(t)|     :  s.t  6  [0.  l]  ,    |s  -t|    <    S     } 

for    X    ^  C    and      S  >  0  , 

Lemma  1.  2  .       If    A    is  a  closed  set  such  that 


(l.lfa)  sup     |x(0)!    <   OO 

and 


(1.17)  lim        sup      M(x,  S    )   =    0  , 

then    A    is  compact. 
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Proof«       It  is  in  the  first  place  easy  to  show  that 

(1.18)  sup       lx(t)|    <    «» 

xe  A 

for  each    t    in  the  unit  interval.     Suppose     {Xj^}CA  ,     By    (1.18)  and 

the  diagonal  procedure  it  is  possible  to  find  a  sequence    {n^}    of 

integers  such  that 

lim      X       (r)   =    L(r) 

exists  for  each  rational    r    in  the  unit  interval.     We  prove  that 

X       (t)   is  uniformly  fundannental.     Given     £  >  0,   choose        S^    so 
n^ 

that    M(x,   S)<e     if     ^<    Sq    and    x€A  .     Then  choose  rational 
To.    •  •  •  •    ^k    so  ^^^^ 

0    =    Tq  <  r,  <  .  .  ,   <  r,     =1 

and    r^-r.    .  <      S^      for    i=l,    ...»   k.     Finally,   choose     N    so  that 

|x        (r.)  -  X       (r.)|    <    £ 
'     n  ^      1  n^      1  ' 

for    i  =  0 k,   provided     ^,v»  >  N  .     It  follows  innnnediately  that 


|x        (t)  -x       (t)       <    3  £ 
n .  J^o 

for  all    t  6  [0,  1],   provided     X,v  >  N  .     Thus    x       (t)    is  uniformly 

n^ 

fundamental  and  hence  converges  unifornaly  to  an  element  of    C  , 
which  must  lie  in    A  .     Therefore    A    is  compact. 
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In  order  to  prove  the  second  of  the  convergence  theorenns  of 
this  section  we  need  a  lenama  which  is  a  slight  variation  on  a  result 
due  to  Donsker     [b]  .     If    x  €  C    and    c    is  a  positive  integer,   define, 
for      i    =1,    .  . .  ,   c, 

a.(x)  =  inf     {x(t)  :  (i  -l)c"^  <   t  <   ic'^  } 

b.(x)  =  sup    {x(t)  :  (i  -l)c"^  <  t  <  jc"^  }      . 

Now  let 

'Tr,(x)   =    (aj(x),    ....   a^(x),   bj(x),    ....   b^(x))  . 

Thus    TT„    maps     C    into    2c-space.     It  is  easy  to  show  that   TT 

is  continuous.     If     t     is  bounded  and  continuous  on    2c -space,   then 

^  (TT  (x))    is  a  bounded  continuous  function  on    G  ,     Let      Ul       be 
c 

the  set  of  functions  arising  in  this  way    (as     c    and     *-f      vary). 

Lemnna  1.  3  .       Let    f    be  a  bounded  uniformly  continuous  function  on 
C  .     Then  there  exists  a  pair  of  sequences  of  functions     (f  }    and 
{f"}  ,   all  belonging  to     (SI    ,   such  that  for  all    c    and    x  , 

(1.19)  f^(x)<f(x)<f;:(x)  , 

such  that    f     and    f"    are  uniformly  bounded, such  that 
c  c  ' 

(1.20)  lim        (f'^(x)  -  fj.(x))   =    0 
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for  all    X  €  C  . 

Proof.      For  each     c     let    M       be  the   (non-empty)  set  of    y   6  C 
such  that    a.(x)<y(t)<b.(x)    for     (i-l)c"<t<jc"    ,    !  =  1,    ...,c. 
And  define 


fj,{x)    =    inf    {f(y)  :  y    6  M^    } 

f^(x)   =   sup    {f(y):  ySMC    }        . 

It  is  clear  that  f*  and  f"  satisfy  (1.19)  and  that  they  are  bounded 
by  the  bound  of  f  .  And  (1.20)  follows  from  the  uniform  continuity 
of    f  . 

There  remains  only  the  proof  that    f     and    f"    belong  to       01.     , 
We  consider  only  the  case  of    f"  ,     Let    S    be  the  set  of  points 
(■i^,,    ....    ^ ->    )    of   2c -space  having  the  property  that 

(1.21)  <.   <   :^^_^.       ,  j  =  l c, 

and 

(1.22)  [  <,.    <c+j^^  f  ^i+1'    ^c+j+1^      ^    °'       ^  =  ^ ^-^    • 

Then    S    is  obviously  closed.     If     (^i.    ...,^-,    )^S    define 

^(<1 ^^^)  =  sup{nY):  <.<y(t)<<'^^..(j-l)c-^ 

<  t  <  ic"    ,     i  =  1,    .  .  .  ,    c}    , 
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where  the  set  over  which  the  supremum  is  extended  is  non-empty 
by    (1.21)  and    (1.22).     Obviously  7r(x)€S    for  all    x€C, 
cp  (it  (x))   =   f"(x),     and     ^    is  bounded.     Suppose  we  piDve  that 
^      is  continuous  on    S  .     Then  it  is  possible  by  Urysohn's  exten- 
sion theorem     [l,   p.  73]    to  extend     t      to  all  of   2c-space  in  such 
a  way  that  it  remains  bounded  and  continuous.     Hence  the  proof 
will  be  completed  if  we  show    ^     is  continuous  on    S  , 

Suppose     (Si.    ...,^->    )^S    and  that    E  >  0    is  given.     We 
will  find  a      S   >  0    such  that 

(1.23)  I  ^(<j <2c^-'''^^l ^  2c^'    <    ^ 

provided 

(1.24)  |<  .-  C  .1    <     S       ,  j  =  1 2c, 

and    (?ii    •...    5t    )€s.     By  the  uniform  continuity  of    f    there 
exists  a      S     such  that     |f(x)-f(y)|    <    ^ /Z     if     ^(x,y)<    5     . 
Suppose  now  that    (  5it    •••»   H   j    )    satisfies  the  above  conditions. 
It  is  clearly  possible  to  find    x    and    y    in    C    such  that  for    j  =  1,    .•.(   c  , 

C  =  inf  {x(t)  :  (i-l)c"^  <  t  <  jc'M     , 

<^^.  =  sup{x(t)  :  (i -l)c"^  <  t  <  ic"^  }     , 

^  .  =  inf  (y(t)  :   (j-l)c"^   <  t  <  jc"^  }     , 

j^  =  sup{y(t)  :   (j-l)c"^  <  t  <  ic*M    . 

^       J 
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Then 

«?(<! ^'zc^  =  ^(-ir^(x))  =   f'^(x). 

and  it  suffices  to  show  that 


(1.25)  |f^(x)  -f;:(y)|    <     £ 


By  the  definition  of    f'      there  exists  a    z  €M       such  that 

c  X 

(1.26)  f^(x)  <  f(z)+   £/2  . 

Now  let    z'(t)   =  z(t)    at  points     t    where 


(j-l)c"^  <  t  <  jc'^        ,  5  .  <  z(t)  <   5 


let    z'(t)   =    5  .      where 

(j.l)c"^  <  t  <  ic"^        .  z(t)  <    5         . 

and  let    z'(t)   =    ?      ,  .     where 

(j-l)c"^  <  t  <  jc"^        ,  ?^..  <z(t)    . 


c+j 


It  follows  that    z'  €M*^  ,    so  that 

y 


(1.27)     f(z')<f;:(y) , 


and  from    (1.  24)  it  follows  that     /^(z,z')<    S    ,  and  hence  by  the 
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choice  of        S     , 

(1.28)  f(2)  <  f(z')  +    £  /Z    . 

From    (1.26),    (1.27)  and    (1.28)   we  have 

r(x).r(y)  <    e    . 

The  symiTjetric  inequality  follows  in  the  same  way,   and    (1.25) 
results.     This  completes  the  proof  of  the  lemma. 

We  come  now  to  the  second  of  the  two  convergence  theorems 
of  this  section.  For  any  integer  c  and  real  numbers  a.,  ....  q^, 
(3.,    ....    j3     ,     consider  the  set 

(1.29)  E  =   (x:  a.  <  x(t)  <  ^.,    (j  -l)c'^  <  t  <  jc"\  j  =  l,    ....  c  } 

Theorem  1.  3  .      Suppose  that  for  probability  measures     P      and    P 
on    ^  we  have     Pj^(E)— ^P(E)    for  all  sets     E    of  the  form    (1.29) 
for  which     P(E)   =    0  .     Then    P    =>    P    . 

Proof.       V  e  show  first  that 

(1. -?0)  {     f    dP^j      -^^      \     f    dP 

for  any  function    f    in       ^      .     For  a  fixed  integer    c    define 


-1 


YJS)    =    P^dr^MS)) 
p  (S)   =    P     (TT^Ns)) 
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for    2c -dimensional  Borel  sets    S  .     Now  if 

S  =   {(<! ^2c^  •   ^j  -  ""j*    ^c+j  -  ^j'    ^"^ ""^   ' 

then  IT  ^^S)   =    E,   so  that      ^'^(S) --^  p(S) ,   provided   U  (S)   =    0  .     But 


this  obviously  implies      U     ^=^  U     ,     Hence 

j^^c  j^<:c 

for  any  bounded  continuous  function    ^    ,     But  if    f(x)   =  *=P  (-TT   (x)  )  , 
(1,30)   follows  frona    (1.31)  by  a  transfornnation  of  the  integrals  in- 
volved.    Hence    (1.30)  holds  for  any  function    i    oi     (Jl    , 

Now  suppose    f    is  a  bounded  uniformly  continuous  function 
on    C  ,     By  Lemma    1,  3    there  exists  a  pair  of  sequences     {f '  }    and 
(f"}    of  functions  of     LR     which  are  uniformly  bounded  and  satisfy 
(1,19)  and  (1,20),     By   (1,20)  and  uniform  boundedness  we  have 

(1.32)  lim  V         (f"    -   f)   dP    =    0  . 

_  1  c  c 


C  -♦-  oo  ^  ^ 


C 

But  from    (1.32)  and    (1.19)   it  easily  follows  that    (1.30)  holds.     And 
since    (1.30)   holds  for  any  bounded  uniformly  continuous  function     f  , 


we  have     P^  rrp»  P  . 


In  the  following  sections  we  will  be  concerned  with  '  random 
polygons"  .  For  the  purposes  of  this  discussion  we  define  a  poly- 
gon to  be  an  element     p    of     C    with  the  property  that  the  unit 
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interval  can  be  decomposed  into  a  finite  number  of  sub-intervals 
over  each  of  which    p    is  linear.     Suppose    {f\.,  (5^,   P)    is  a 
probability  measure  space  on  which  random  variables    X^,   X,,  .,,,  X 
are  defined.     Associated  with  each  point   co    of  S\  we  define  a 
polygon    p  =  p         by  the  equations 

(1.33)  p^(t)=  {j.nt)X.^^(c.)+(nt-j  +  l)X.(<^),    (j-Dn^^^t  <  jn"^    . 

j  =  1,    ....   n  , 

that  is,     p    is  the  polygon  with  vertices  at  the  points    (jn"  ,   X.)  , 
We  will  be  interested  in  the  distribution  properties  of    p  ,   that 
is,   we  will  be  interested  in  probabilities    P{p  €  A}  ,  where    A  €  C  , 
In  order  to  show  that    P{p  €A}    has  meaning  we  must  show  that 
{cj:p     €A}e(S,i,e.,  that  the  mapping    «*>  — ^  p,^      is  measurable. 
Since  the  collection  of    A's    for  which  this  holds  forms  a  Borel 
field,    it  suffices  to  prove  it  for    A's    of  the  form    A=  {x:x(t)  <  a}   , 
But  if    A    has  this  fornn,  and    (j  -l)n"    <  t  <  jn"    ,     then  by    (1.33) 

{a>:  p^€  A}    =    {to;   (j -nt)X.^j(a>)+ (nt  -  j +  l)X^(u>)  <  a}  , 

which  lies  in  O     since  the    X.    are  measurable. 

We  can  use  this  result  to  set  up  on  t  a  probability  measure 
P*  which  gives  unit  mass  to  the  set  of  polygons  which  are  linear  on 
each  of  the  intervals     ((j-l)n"    ,   jn"),   j  =  1,    ....   n,     and  under 
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which  the  random  vector     (x^  ,   x      i  ,  x_      ,  ,    . ,  ,  ,   xj    has  a  pre- 

scribed  distribution  function.     In  fact  if    X^,    X  ,    .,,,    X      are 

random  vectors    (on  some  probability  space    {H.,  Qo  ,    P) )    having 

the  prescribed  distribution  function,  and  if    P'    is  defined  by 

P'(A)   =    P  {p  e  A}    for    A  €   t  ,    then    P'    obviously  has  the 

desired  properties, 

A  useful  fact  in  the  theory  of  distributions  on  the  real  line 

is  that  if  the  distributions  of  a  sequence     {X    }    of  randonn  variables 

converge  weakly  to    F  ,    then  so  do  the  distributions  of     {X    +Y    }  , 
o  /  »  n        n 

provided      p  lim      Y      =  0  .     We  conclude  this  section  with  a  theorem 
which  plays  an  analogous  role  in  the  theory  of  distributions  on    C  . 
The  theorena  and  its  proof  obviously  remain  unchanged  if    C    is 
replaced  by  any  Banach  space. 

Let     {  X^  }    and     {  Y     }    be  two  sequences  of  nneasurable 
functions  on  sonne  probability  measure  space     (Jl  ,  CS  ,    P),     with 
values  in    C  .     That  is,    we  assunne  that    X^(w)  €  C    for    co   6  X\ 
and  that     {  o  :  X^(w )  €   A  }  e  ©    if    A  e  fe     ,     and  similarly  for    Yn  . 
Let    Pn  ,    P^    and    P^    be  the  distributions  on    C    of    X^  ,    Y^  and 
X^  +    Yj^      respectively: 

P^(A)   =    P{X„    6    A}     . 

P;^(A)   =    P{Y^  €    A)    , 

P;^(A)   =   P{X^+Y^  6  A}     , 
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for    A  €  C  ,     Let    U    be  the  measure  on    t   which  places  unit  mass 
at  the  function  which  is  identically  zero. 

Theorem  1.4.      If    P„  =>  C    and    P'   =^U    then    P"  =:>  O  . 
n  n  ^^  n 

Proof.      It  is  clear  that    P^  =>  U    is  equivalent  to  the  statement  that 

(1.34)  lim      P    {/?(0,  Y„)>   £  }  =  0 

for  all     £  >  0  ,     Let    f    be  a  bounded  uniformly  continuous  function 

on    C  .     Given    ^   choose     S    so  that      |f(x)-f(y)|   <  ^    if     P(x,  y)  <    ^  , 

Then 

P{|f(X^).f(X^+Y^)|   >    £  }   <  P{^(X^.   X^  +  Y^)>    S    }-^  0 

by    (1.34),     Hence 

(1.35)  p    lim      (f(Xn)-f(Xn  +  Y^))  =  0  . 

Now  by  Theorem  1.1  ,      P  {  f(Xj^)  <  a} ->•  Q  {x  :  f(x)  <  a)    at  continuity 

points  of  the  later  function.     Hence  by    (1.35)  and  the  above -nnentioned 

fact  in  the  theory  of  distributions  on  the  real  line, 

P  {f(X    +Y    )  <  a)-^0  {x  :  f(x)  <  a}  .      Since    f    is  bounded  this 

implies 

^  f  dp;;        — ^       I  ^  ^°  • 

c  c 

Hence  by  Theorem  1.1    ((ii)-»-(i))    we  have     P^  ^  C  . 


-•  so  . 


§2.     The  existence  of  Wiener  measure. 

A  fundanmental  problem  connected  with  the  measure- 
theoretic  aspect  of  probability  theory  is  that  of  proving  the  ex- 
istence of  stochastic  processes  having  specified  properties.     In 
this  section  we  give  a  proof,    based  on  Theorem  1.  2,    of  the  ex- 
istence of  the  Wiener  process. 

Aprobability  measure    P    on  C  is  called  Gaussian  if  the 

P-distribution  of  the  random  vector     (x     ,  .  .  .  ,  x      )    is,   for  any 

1  n 

set  (t  ,  .  .  .  ,  t  )  of  points  of  the  unit  interval,  a  normal  distri- 
bution with  zero  ntieans.  For  such  a  measure  P  the  covariance 
function 


R(s.t)  =     \        X   X     dP 
^  C      ^ 

is  defined  for  all    s    and    t    in  the  unit  interval.     Moreover    R 

completely  determines    P   .     In  fact    R    determines  the  covariance 

matrix  and  hence  the  distribution  of  each    (x    ,  .  .  .  ,  x      ),     and 

1  n 

these  in  turn  clearly  deternnine    P   .     The  question  is,  given    R, 

does  there  exist  a  Gaussian  measure     P    with    R    as  its  covari- 
ance function?     The  following  theorem  gives  an  affirmative 
answer  in  the  case    R{s,t)  =  nnin  (s,  t )  .     The  Gaussian  measure 
having  this  covariance  function  is  called  Wiener  measure,    and  is 
denoted  here  by     W   .     It  is  a  simple  matter  to  show  that  if 
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t-  <  t^  <  •  •  •  <  t    ,     then  the  random  variables 
12  n 


X  "    X      ,     X  "•    X  I    .  «  •  ,     X  "    X 

^2         h       ^3         ^1  ^n  n-1 


are,   under     W   ,    normally  and  independently  distributed  with 
zero  means  and  variances 


2        13         2  n        n-1 


Theorem  2. 1.    There  exists  a  Gaussian  measure    W    such  that 

(2.1)  \       X  X.    dW  =  min  (s,t) 

J  ^     s    t 

for  all    s    and    t    in  the  closed  unit  interval. 

Proof.     For  each  positive  integer    n    let    A     €  O  be  the  set  of 

polygons  which  are  linear  on  each  of  the  intervals 

{( j  -  1  )Z'^,   j2""),   j  ^-  1 2  "    .     Let    P       be  a  probability 

measure  on  ^  such  that    P    (A    )  =  1    and  such  that  the 

n      n 

P    -distribution  of 
n 

(2.2)  (x    ,    X  „  '   ^  ,  .  .  .  ,    X    ) 

0        1^2"  2-2"  1 

is  normal  with  zero  means  and  covariance  matrix    (  A..  ),     where 

A..  =2*'^  min    (i,i).     That  such  a    P       exists  follows  from  the 
ij  \  '  J  /  j^ 

rennarks  preceding  Theorem  1.4    and  the  fact  that  the  matrix 
(A-)    is  positive  semi-definite  (in  fact 
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^      \  -n       ^"         2"  2 

/\ .  .u.u.  =  2        L.      '  ■" 


Z..       A   .u.u.  =  2        L.      (  S.  .    u.  )     >  0  .  )    Note  that  under    P 
the  differences 


(2.3)  X  -   X       ,     X  rx    '    ^  n    '      •  •   • 

1-2""         0         2-2'  1-2' 

are  independently  and  normally  distributed,   each  with  variance    2" 

We  will  first  show  that    P       satisfies  the  hypothesis  of 

n  '  ^ 

Theorem  1.2.     For     0<^<1    let    (3  (  S  )  =  -  Ig  S        .     Then 

(2.4)  P  (  S)  I  as    S    t    , 


and 

(2.5)  S  {3(  S)  t  as    S    t     , 

provided     ^    <   T    •     Given  an    £  ,   with    0  <   £  <  A     ,   let    Kg      be 
the  closed  set 


{  X  :  sup        (3  (  S  )  M(x,  S  )  <    A/£    }   , 

o<  %<  i^ 

where    A    is  a  positive  constant  to  be  determined  later.     Then    Kg 

is  compact  by    Lemma  1.  2,    since    (3*    (  S  )    ]f    0  as    S  |  0  .     We  will 

prove  that    P    (Kg)  >  1  -  £    for  all    n    by  showing  that 

(2.6)  P     {  x  :        sup         p(  8  )  M(x,  S  )  >   A/ 6   }  <    £     . 

If    x    €   A       it  follows  from     (2.5)  that 
n 
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sup  P(S)M(x.  S)=      sup  3(S)—      max       |x((k+l)2'*")  -  xikZ""^) 

=     p{^"*^)      max       |x({k  +  l)2'")  -  x(k2"")|     . 

Hence,    since    P    (A    )  =  1  ,    since  the  differences     (2.3)    are  normal 
n      n 

and  since        €   <  A"     ,   we  have 


(2.7)         P     (      sup         p  (  ?)  M(x,  8  )  >  A/£  } 

2^-1 

<       IZ     P„{P(2"'')  |x((k+l)2-").x(k2-'')|>   f-}. 
n  ^ 

k  =  0 


£      ^/T    ,    ,       ,n/2  .  1  2"      .   _A^   . 


<      ~    A'     . 


where 


A'  =     sup  j^  igZ  n  2""/^    exp(  -        ^  .        '   " z"  )  <     *=^     • 
T»>1    '^  2  Ig    2  n 

On  the  other  hand  from    (2.4)    and  the  fact  that    M(x.  S"  )    y    as 
8   t        ,    it  follows  that 

(2.8)  sup  p(S)M(x,  8)=       max  .sup  p(S)M(x,8) 

<        max         p(2J"^"")M(x,2-^""^     ' 
2^  iin-z 
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But  if    X  €  A     ,   a  moment's  reflection  shows  that 
n 


(^..9)  M{x.  2^"'') 


<3     max         sup      {  |x(t)  -  x(  12-^"^  )  | :  iZ-^'^l  t  <    {i  H  )2^"""  } 
o<i<z"-J 


=  3     max    .     max    .         |x((i2J  +  k  )2"'^)  -  x(  12^"")  |     . 
o <  i <  z"'^    o<k<z^ 

Now  by  a  well-lmown  inequality  concerning  the  maximum  of  the 

partial  sums  of  iiidependent  symmetric  randoni  variables 

[8,    p.    106  j     we  have, 

(2.10)  P     f  3f^{2J  "^"'^)    max     |x((  i2^+k)2"'^)  -  x(  12^"^)  |  >  A/  £     ) 

<    Z   P^{36(2J"^"")|x((i+l)2J"'')  -  x(i2J"")|  >    A/e    } 

^    J-      1"  "^ZL    n-i  +  I  ,         L 2""-^ 

-    A  A72Tf  .{n-jlT^     ®^P^-      ,_  ,    2-         ,      .^.,2       ^     • 

2       ■'  '  18  Ig    2         (n-j+1) 

From     (2.8).    (^.  9)    and     (2.10)    it  follows  that 

(2.11)  V      {         sup  p(  S)M(x.  S  )>     A/  £    }  <    -^    A"    . 

where 

^"    =  "^7#       -        2^^    (k+1)  exp  (. ^~-j-     — ^-— )  <    oo   . 

^'"^       k:^2  18  lg'^2         (k+1)^ 


IJ  we  take     '\    -  A'  +  A"     then     (2.6)    follows  from     (2.7)    and     (.'.11) 
We  next  show  that 
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(2.12)  \      X  X    dP     -^min  (s,t)  . 

J^.     s    t        n 

Suppose  that    s  <  t    and  let    j     =  [s2   ]  and  k     =  [t2    ].     If  x  ^  A       then 


n       '        '  n       *^        '  n 


x(s)  =  x(j^2-^)+(s2"-j^){x((j^  +  l)2-'')  -x(j^2-"))  . 


x(t)  =  x(k^2-")  +  (t2''  -  k^)(x((k^  +  DZ-"")  -  x(k^2-^)) 


Hence,    since  the  differences     (2.3)    are  independent, 


J 


X   x^  dP     =   \      x(j    2"*^)  x(k    2"^)  dPn  =  j    2"" 
J-,      s    t        n       j„      ^-"n  ^   n  •'n 


Since      {P     }     satisfies  the  hypothesis  of  Theorem  1.2,     there 
exists  a  sequence  {n     }  and  a  probability  measure    W     on  C   such 
that    P       =^W  .     It  follows  easily  from  the  normality  of  the 
variables     (2.2)   that    W    is  Gaussian.     And    (2.1)    follows  fronn    (2.12) 

The  method  of  proof  of  this  theorem  can  be  applied  equally 
well  in  other  cases.     It  is  possible  to  use  it  to  prove,   for  example, 
the  existence  of  the  stochastic  process  which  arises  in  connection 
with  Doob's  approach  to  the  Kolmogorov-Smirnov  theorenniS  [7]  . 

We  note  at  this  point  the  well-known  fact    [8,  p.  392]  that 


-       *  2. 


W{x:      max      (x(t)  -  x(a))  >    \     ) -^f^rt^    [     ^.u/2(b-a)    ^^  ^ 
a<t<b  '    ^  ^^'^'     -^)^ 

It  follows  from  this  that  the     W-measure  of  the  boundary  of  any 

set  of  the  form    (1.29)    is  zero. 
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S3.     A  general  invariance  principle. 

In  this  and  the  following  sections  we  will  be  concerned  with 
proving  the  convergence  to     W    of  the  distributions  (on   C)  of  cer- 
tain sequences  of  random  polygons.     Let    X.,X^,  ...       be  a  sequence 
of  random  variables  on  a  probabaility  space     {jTl,   Qb  ,  i"*  )•     iet 
p       be  the  random  polygon  defined  by 


n 


Pn^^^  "  ^j-1  "^  ("t-J  +  l)Xj   '    (  J-1  )n'^  <   t  <    jn"^  .     j  =  1 n, 

where    S.  =  X,  +  •  •  •  +  X.    and    S^  =  0  .     Thus    p      is  the  polygon 
jl  J  0  '^n  r/b 

with  vertices  at  the  points    (jn"  ,    S.).     As  remarked  at  the  end  of   §1  , 

P   (p     €  a}    is,  for    A   6  C«    ,   a  well  defined  quantity.     Suppose  there 

exists  a  sequence     (a     }    of  positive  constants  such  that  if  the 

measure    P      is  defined  by      P    (A)  =  p{a"p     €A}    then    P    =^  W  . 
n  n  n     n  n 

If  thia  is  true  we  say  that  the  invariance  principle  holds  for  the 

sequence     {X     }     with  norming  factors    a      .     It  is  the  purpose  of 

this  section  to  derive  a  general  set  of  conditions  on      {  X     }    under 

°  n 

which  the  invariance  principle  holds.      What  the  conditions  lack  in 
elegance  they  make  up  for  in  utility.     In  the  subsequent  sections  we 
specialize  these  conditions  in  various  ways. 
For  integers    c  ,  \>   and    n    define 

n    =  [ jnc'   ],  j  =  0 c, 

•I 

n.   „  =  [n(V>   (j-l)  +  u)c"    V  "    ],   j  =  l,  ...,c  ,   u  =  0     ...,V 
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For  any  real  numbers    a.,  S.    with    a.  <   (3.  ,  j  =  1 c  ,     let    E 

J       J  J  -     J  n.  r 

be  the  S\    set  where  the  relations 

(3.1)  a.  <  a"    S.  <  (3.  if      n.      ,  <  i  <  n. 

J  -     n       I  -  ^j  J  - 1  -     J 

are  satisfied  for    i  <  r  ,   but  not  for    i  =  r  . 

Theorem  3. 1.      The  invariance  principle  holds  for  the  sequence 

{  X     }    with  normine  factors    a       if  the  following  two  conditions  are 
^     n  ^  n  ^ 

satisfied. 

Condition  (i  )  .     For  each  integer    c    the  distribution  of  the  random 

vector 

(3.2)  a"^  (  S       ,   S       -  S       ,  .  .  .,   S      -  S  ) 

n  n,  n-         n,  n  n      , 

12  1  c  c-1 

approaches,    as    n  —*-  oo   ,   the  normal  distribution  having  zero 

means  and  having  as  covariance  matrix    c"      times  the    cxc    identity. 

Condition  ( ii  ).    For  each  integer    c  ,   each  set 

(a,   ,  .  .  .  ,   a    ,    6, p     )    and  each    £  >    0  , 

1  c        1  c 

(3.3)  lim        lim  sup      Z)      P(E  t-\  {  \  S  -S|>£a})       :^0. 
V-«-«>      n-^  ««        |*a|               ^'^                 ^j,u+l         ^  ^ 

where  the    n  ,     corresponding  to  each    r    is  defined  by  the  re- 

J,u+1 

lation 


(3.4)  n.       <  r  <  n.       .  ,     . 

^  j,u  -    j,u+l 

Proof.     Throughout  the  rest   of  the  paper  we  will  be  dealing  with  sums 
of  the  sort  appearing  in     (3.3)  .     In  each  instance    n.  ,     is  a 

function  of    r     defined  by     (3.4)   . 
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To  prove  the  theorem,   let    E       be  the  TL    set  where     (i.l) 


n 


is  satisfied  for  all    i  =  1 n  .     Then  n  -  E     =   \J^_    E 

Let    E    be  the    C     set  where 


Q.  <  x(t)  <  S. 
J  ~  ~    J 

if    (j  -  l)c"    <  t  <  jc"     ,   for    j  =  1  ,  .  .  .  ,   c  .     Let    D  ^    be  the    C 
set  where 

a.  <  x(((j-l)  V    +  u)c'^  V  "S  <  p. 

for    j  =  1 c    and    u  =  1  ,  .  .  .  ,    V     .     Further,   let    F      be  the 

n 

St      set  where 

a.  <  a"   S  <  (3. 

J  -     n      n.        -     1 

for    j  =  1  ,  .  .  .  ,    c    and    u  =  1  ,  .  .  .  ,    V*      .     Finally,    let    E  ^         ,     D^  ^ 

and    F  be  defined  in  the  sanne  way  as    E,    D_x      and    F    ,    but 

n, e  '  ^  n 

with    a.    and    (3.     replaced  by    a.  +  6     and    p.  -  £     respectively. 

For    n.        <  r  <  n.       ,,     write 
j,u  -     j,u+l 

(3.5)    P(E        )  =  P(E  rx    {|S  -S    |  >     £  a    }) 

n.r  n.r  n.^^^^        r 

+  P(E  nils  -S|<a      c}). 

*    n, r  '    n.       , ,        r '  n 

J,u+1 

Obviously  the  set  in  the  second  term  of  the  right  member  of    (3.5) 

is  contained  in  S\  -  F  .     Hence,    since  the     E  are  disjoint, 

n,  £  n, r 

we  have 
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n  ^ 

1  -  P(E   )  =      2  P(E        )  <  1  -  P(F           )  +      S 

"          r=i  n,  r'  -             ^    n,  £    '             >».  n    ' 

where        C           is  the  sum  in    (3.3)  .     Since    E      C  F       nac  have 

^  V,  n  '         '                           n             n 

P(F           )  -  ^,       <  P(E    )  <  P(F    )  . 

'    n,£    '  ^T?,  n  -      *    n     -      ^    n 

But 


lim        P(F^)  =   W(D„  )  . 


lim       P(F      -    )  =   W(D  )  . 

by  Condition     (i)  .     Hence 

W(D    ,         )  -    lim  sup       C  <    lim  inf    P    (E    )  <  lim  sup  P    (E    )  <   W(D,  ) 

V,  e  ti-^»-e»  ^  x>,n  -  x\-^vo        n      n-^_^^        n      n-  v 

Letting    V  — >-  co    we  have,    by  Condition  (ii)  , 


W(Ep  )  <  lim  inf  P(E    )  <  lim  sup  P(E    )  <  W(E)   . 

Now    E  .    t   E°    as  £  io  .     Since    W(E)  =  0,    lim   P(E    )  =  W(E)  . 

The  proof  of  the  invariance  principle  will  be  coinplete  if  we 

prove  the  following  lemma,   the  hypothesis  of  which  we  have  just 

shown  to  be  satisfied. 

leiTima  3.1.       Let    A     be  the  set  of  polygons    p    which  are  linear 

on  each  of  the  intervals     ((i  -  l)n      ,  in     )    and  satisfy    p(0)  =  0  , 

and  suppose     P       is  a  measure  on    v3   with    1  (A    )  -   1  ,     Suppose 
'  ^  n  n 

further  thit 
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(3.6)  p^(GJ-^-W(E) 

if    E    is  any   set  of  the  form    (1.29)    and    G      is  the  set  of 
X  €  C    for  which 

a,  <  x(in"  )  <  S. 

if    n.    ,  <  i  <  n.  ,   for    i  =  1  ,  .  .  .  ,  n  .     Then    P     =^  W  . 
J-1         -    J  n  ^ 

Proof.     Let    £  be  a  small  positive  rational  and  let    Eg      be  the 
set  where 


aj<x(t)<Pj 
if      (j-^''"'   +   £  Itljc"   -€    (j  =  2  ,...,  c),  where 


aj<x(t)<  p^ 


0  <  t  "    -    £  ,   where 


a    <  x(t)  <  p 


1-c'     ^6<t<l,   and  where 


max  (  a.,   a.^j)  <  x(t)  <  min  (p  ,    p  ^^) 

ii    jc'     .  S  <  t  <  jc    +   e      (j  =  1  ,  .  .  .  ,   c  -  1 )  .     Analogously, 

define    G      „      to  be  the  set  where  for     i  =  1  ,  .  .  .  ,   n  , 
n,  s 

a.  <  x(in"  )  <  p. 
J  -  -    J 
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if    (j   -  l)c"    +    £    <    in'    <  jc"    -     €  ,    (j  =  2  ,  ,  .  .  ,    c  -  1 )  ,   where 

a.  <  x(in     )  <  p, 

if    0  <  in"    <  c"     -   £  ,    where 

a     <  x(in"  )  <  (3 
c  —  c 

ifl-c       +€'<in      <1,    and  where 

max  (  a . ,    a .  . ,  )  <  x  (in     )  <  min  {  6 . ,  3 .  ,  ,  ) 

if    jc'     -   £  <  in'    <  jc"    +    £       (j  =  l c-1).     Since     £    is 

rational,     E-     can  be  cast  in  the  form  (1.29)    and    G  bears 

•^  n ,  e 

the  same  relation  to    Eg    as    G      does  to    E.     Then  by  hypothesis 

(3.7)  P    (G^       )   —^    W(Eg  ) 

n      n,  c  * 

as     r\  — »"  «>       .     Now    E    C  G    ,     while    G         ACE    provided 

n  n,£      n 

n'    <    €       .     Hence  x 


P„(G^    ^    )<  P„(E)<P    (G^) 
n      n,  c  n  n      n 


for  large    n    and,    by     (3.6)    and     (3.7), 


(3.8)  W(E_)<     lim  inf   P    (E)  <    lim  sup     P    (E)  <  W(E). 

^  n  —*-  oo        n  x\  — >-  oo  n 

Now    Eg  I  F    as     £  |  0   ,    where     ^V(E  -  F)  -  0   .      Letting    £  -^^  0 

in    (3.8)    we  have      lim    P    (E)  =  W(E)  .     Hence    P     r^   W  by 

ti-^oo       n  n 

Theorem  1.3.     This  completes  the  proof  of  the  lemnma,    and  hence 
of  the  theorem. 
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With  only  slight  complications    Lemma  3,1    can  be  proved 
with     W    replaced  by  an  arbitrary  linniting  nneasure. 
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S4.     The  invariance  principle  for  Markov  processes. 

In  this  section  we  prove,   using  Theorem    3.1,    the  invari- 
ance principle  for  discrete  Markov  processes  satisfying  Doeblin's 
condition.      We  use  the  definitions,    notations  and  results  of 
(8,    Ch,   V]  . 

Let    X    be  a  space  of  points  ^    and  let    -y-      be  a  Borel 

field  of  subsets  of    X  .     Let    {  x    ,    n  >  1  }    be  a  Markov  process 

n         — 

with  state  space    X    and  stationary  transition  probabilities 

(4.1)  P(?.A)  =  P  {  x^^^€A||  x^  =^  }  . 

That  is,    {  X     }    is  a  sequence  of  measurable  functions  from,  some 

probability  space  (Jf7  ,    ^,    F)    to    X  ,    such  that    (4.1)    holds, 

where  the  transition  function    p    is  a  measurable  function  of    ^ 

for  a  fixed    A    ^    -f       and  is  a  probability  measure  on    ~/^ 

X  X 

for  fixed     ^    .     The  initial  distribution  'TT    is  defined  by 

7T  (A)  =  P  {x^  e  A  }  , 

and  the     n-step  transition  probabilities  by 

p<^N^.A).P   (x^^^e  All  x^.   ?    }   . 

The  existence  probleins  involved  here  are  resolved  in    [8]     . 
We  assume  that  the  process  satisfies  the  hypothesis  of 
Doeblin: 
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Hypothesis    (D).      There  exists  a  finite-valued  measure 

on      -/t.  ,   an  integer    V    >  1      and  a  positive      £    ,    such  that  if 

SP    (A)     <     B      then 


P^**^  (f.A)<l  -  e 

for  all   ?    €  X  . 

Note  that    (D)    is  a  hypothesis  on  the  transition  function 
alone  and  is  independent  of  the  initial  distribution        "TV 

It  is  shown  in   [8]    how,  under    (D),     the  states    ?     can  be 
classified  according  to  their  ergodic  properties.     It  is  shown 
further  that  the  ergodic  theorem  holds  if  the  following  hypothesis 
is  satisfied. 
Hypothesis     (D    )  . 

(a)  Hypothesis    (D)    is  satisfied. 

(b)  There  is  only  a  single  ergodic  set  and  this 
contains  no  cyclically  moving  subsets. 

That  is,   it  is  shown  that  if    (D   )    holds  then  there  exist  positive 
constants   V"    and     /^     ,      /^    <  1,   and  a  (unique)  stationary  initial 
distribution    p    such  that 

,(n) 


Ip^^^^f    ,E)-p(E)|  <  >^^    " 


for  all     f    €  X  ,    E    ^   ^      and    n  >  1     .     The  results  of  this  section 
will  be  obtained  under  the  assumption  of    (D    )     . 
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In  what  follows  the  initial  distribution  under  the  assumption 
of  which  a  probability  is  computed  will  be  denoted  by  a  subscript, 
thus:    P      (E)  .     If  -TT  =  p  ,   the  stationary  initial  distribution,    the 
subscript  will  be  omitted.     In  statements  involving  only  transition 
probabilities,    e.g.  ,    (4.1),    the  initial  distribution  is  irrelevant 
and  the  subscript  will  be  omitted  in  any  case. 

We  state  for  reference  three  results  proved  in    [8, p.    224]  . 
Lemma  4.1.       Under  Hypothesis     (D    ),   if    f    is  a  bounded  (perhaps 
complex-valued)  random  variable,      |f|  <   M  ,     on    x,     .,   x,     ^,.., 
sample  space,   then 

(4.2)  |E  (f   ||x^}  -  {f  }  I  <  2    >-  M^^  . 

In  several  of  the  subsequent  applications  of  this  lemma    f    will 
be  the  characteristic  function  of  a  set. 

Lemma  4.2.      Under  Hypothesis     (D    )  ,   let    f    be  real-valued  function 
of    §     ,    measurable     ^^  ,   with 

E{f(x^)}  =  0,       E{(f  (x^  )^}  =  or^  <     «x>     . 

Then  as    n  — »-  oo    , 

n  , 


E{(      E      f(x.))''}^n   <r,        , 


2 

where       CT       is  a  constant  depending  on    f    and  on  the  process 
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Lemma  4.  3.      Under  Hypothesis    (D    ),    let    f    be  a  real-valued 
function  of     ?    ,    measurable    7*^  '   ^^^^ 

E  {f(xp}  =  0  ,       E{  |f(Xj)|^"^M  <     ^ 

for  some     o   >  0  .     Then  there  is  a  constant    a  ,   for  which 

E{|     Z     £(x.)|2^Mian'+(*/^',  n=l,Z 

j  =  l        > 

It  is  convenient  to  have  available  the  following  corollary 
of  Lemma  4.1  .     Suppose  we  have  positive  integers    u.,v.     with 

(4.3)  u    <  V    <  u     <  V     <  ...  <  u      <  V        . 

i         I         c         d  mm 

Suppose  further  that 

(4.4)  u.-v.,>B>l,  i  =  2,...,m. 
'                                  1         1-1  —       —      ' 

Lemma  4. 4.      Under  Hypothesis     (D    ),    let    f.    be  a  (perhaps  com- 
plex-valued) random  variable,   with     If.  I  <  1,     on    x     ,  .  .  .  ,x 
'^  '  J  '   —  u.  V. 

sample  space,   for    j  =  1,  ,  ,  .  ,  m     .     If    (4,3)    and    (4.4)    hold,    then 

(4.5)  |E{fj--.f^}  -  E{fj}..-  E{f^}  I    <    2m-K^®    . 

Proof.       The  proof  goes  by  induction  on    m  .     The  result  being 
trivial  for    m  =  1  ,   assunne  it  is  true  for  some    m  -  1  .     Then 

E{f-"f      }  =  E{f/--f       ,}E{f       11  X  }  =  E{f,---f       ,}E{f     }  + 

'^    1  m  ■'  ^  }  m-1        ^  m  "      V       ,  1  m-1  m"^ 

m-i 


S  . 
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where     |  €  |  <  2  T p         by  Lemma  4.1  and  the  bound    jf.  |  <  1  . 
And  now    (4.5)    follows  from  the  induction  hypothesis. 

We  come  now  to  the  invariance  principle. 
Theorem  4. 1.     Under  Hypothesis     (D    ),   let    f    be  a  rea'-valued 
function  of    ^    ,     measurable     "/V'    with 

E{f(xp}  =  0  .       E{  |f(xp|^'^^}  <    oo 

for  some     8  >  0  .     Then 

(4.6)  lim    E{  (n"  ^^        S      f(x.)  )^}  =   CT^ 

n-^«>  j  _  2  J  i 

2 
exists.     M    ^.     >  0    then  the  invariance  principle  holds  for  the 

1/2 
sequence     {f(x   ))    with  norming  factors        (T.n  '      ,    no  naatter 
n  1 

what  the  initial  distribution      TT  . 

Proof.       That  the  limit    (4.6)    exists  is  simply  a  restatement  of 
Lemma    4.2  .      We  prove  the  result  first  under  the  assumption  of 
stationarity  and  remove  this  restriction  later. 

We  must  show  that  Conditions  (i)  and  (ii)    of  Theorem  3.1 
are  satisfied.     In  the  notation  of  that  theorem,    we  must  first 
prove  that  the  distribution  of  the  vector 

-1  ^-1/2  ^g_s       -S,...,S       -S  ) 

1  n^      n^         n^  n^         n^.^ 

approaches  the  appropriate  normal  distribution,   where 
S,    =  f(x.)  +•  •  •+  f(x,  )  .     Our  proof  of  this  part  of  the  theorem 
follows  [8]  .     Let     {OC^}     and     {  p    }    be  two  sequences  of  positive 


-  48  - 


integers  such  that  if     {  P    }     is  defined  by 


-1 


(4.7)  y^-[     (min       ("•   -  n.^^)  -  PJ(a^  .  PJ       ] 


then 


(4.8)  lim       H     (3     a    "^  =  0  , 

n-^oo      /  n     n     n 

(4.9)  lim      p      p    "  :.  0  , 

while    Q    ,  (3       and    U        all  go  to  infinity.     For  example  one  can 
n      n  r  n  '' 

4-1  3 

take     0       '~'  no        and    a     =  0 
n  n       '^n 

Now  for    j  =  1,  .  .  .  ,  c  let 

y.  =  /_,  f(x  .  •  )    .  m  =  1,  .  .  .  ,    M 

'j,  m  * — '  n.    ,+1  I 


y?         =  y^  f(x  .  .   )    .  m  =  1 H 

'j,  m  I — ^  n.    ,+1  \ 


J.  I  is  i;<.(+p)-H  J-1 

We  prove  that 

-1/2      ^+^ 
(4.10)  p  lim    n    '/^        S       y!  =  0  ,  j  =  1,  .  .  .  ,c 


n  ->■«» 


m  =  l       J'"^ 
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Now  by  Lemma  4.2,    Minkowski's  inequality  and    (4.8), 


rv^i        J.m  '    '  ^_,  ^ "  J.m 

nT=l       ''  nn=  1 


<   An-V2  (^pl/2  +  (a  +  Zp  )^/^)  <    A  ^"^/^  q"^/^  (^^p^/^  +  (  a  +  2p)^'''' .   -^0  , 

where    A    is  a  constant.     This  implies    (4.10)  . 

Hence  by  Theorem    A.  4    it  suffices  to  prove  the  asymptotic 
normality  of  the  random  vector 

(4.11)  cr-^n"^/^    (       2        y,    ^.....        Z      y  )     . 

1  m  =  l      ^'"^  m=l      ^'"^ 

Let 

f   (u  .,,..u   )  =  E{exp(i       E      u         S      <^i"^  n"^/%.     ^)  ) 
'^     ^  ^  j  =  l       J    m=l       ^  J''" 

be  the  characteristic  function  of    (4.11)  .     Now  the  last  term    f(x.) 

occurring  in    y.  and  the  first  occurring  in    y.        ,,    have     3 

such  terms  in  between  them.     And  the  last  term  of    y.  ^    and  the 

first  of    y.,,    ,    have  at  least    S    others  in  between.     Hence  by 
'j+1,1 

Lemma  4.4, 

I'  c 

m  =  lj  =  i  J     J' 

where    |€    |  <    Z^c  |^P  ^''"^-^0    by    (4.9).     Let 
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z.  (j  =  1,  .  .  .  ,  c,    m  =  1,  .  .  .  ,  P    )    be  independent  random 

j.m  r 

variables  each  having  the  distribution  of    y,    i   •     By     (4.12),    the 
proof  of  Condition  (i)  will  be  complete  if  we  show  that  the  dis- 
tribution of  the  vector 

-1     -1/2  ^  ^ 

(4.13)  a-'n'^'-i       Z      z  ^    ^^    rJ 

^  m=l       ^'"^  m=l     ^•''^ 

approaches  the  appropriate  normal  distribution.     Since 
Y  /an-^c'      by    (4.8),   it  follows  that 

(4.14)  lim      E  {{  <r'^  n'^^^        Z       z.        )^  }  =  c"\  j=l c 


m  =  l 


Since  the  components  of    (4.13)    are  independent,    its  covariance 
matrix  approaches    c"      times  the    cxc    identity  matrix.     By 
Theorem  A.  2    it  suffices  to  show  that    Lyapounov's  condition  is 
satisfied.     By  Lemma  4.3    there  is  a  constant    a     such  that 

iTj  I  l^+S   \  ^         l+(5/2) 

El    z.  }  <  a  a     ^    '     ' 

'    j,m  •  '  - 

Hence,   by     (4.14),   for    n    sufficiently  large, 

1  1  J.m'  C/o 


E{(<ln-l/^        2      z.        )2})l+(^/^) 


m-  1 


Thus  Lyapounov's  condition  holds. 

Having  shown  that  Condition  (i)  of  Theorem  3.1    is  satisfied, 
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we  turn  to  Condition  (ii).     Let    E         ,   n.       . , .     c  and    V    be  as 

n,  r       j,u+l 

in  that  theorem.     Define  a  sequence   {p    }    of  integers  by 
Pn  =  [Ig  n]  .     If    r  +  p<  n.^^^j    then 

(4.15)  P(E^    .aHS^  -  S    I  >   En^^}  ) 

^'"^  ""j.u+l 

<  P(E  ^{|S  ~-  S   ^„|  >     £  n^^/l  }  ) 
—       *    n,r  ^     '    n.       ,,  r+S '  —  /      *   > 

J.u+1 

and  we  can  estinnate  the  terms  on  the  right  separately.     Now 

(4.16)  P{|S^  -Sr+3l>     en^^V^llxj x^  } 

j,u+l 

<  P{|S  -S   ^-1  >    en^/V2}+  2T-A>P 

•^j.u+l         ^+P  ' 

by  Lemma  4.1  and  the  Markov  property.     And  by  Chebyshev's 

inequality  and  Lemma  4.  2» 

j,u+l        r+p'  - 

<  A  6"^n"^(n.   ^^j-  (r+p)  )  <    2A/e^c   »>     , 

where    A    is  a  constant.     By  (4.16),    (4.17)  and  the  defining  property 
of  conditional  probabilities, 
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(4.18)  P(E  r\{\S  -Sa.ftl>     S,n^^/Z)) 

J , u+1  / 

<  (2A ^  ^  ir^P)t{E^  ^)     . 

^     C   V 

To  estimate  the  second  term  in    (4.15)    observe  that 

(4.19)  ^nS^^p.Sj  >    tn'^/2.) 

r  +  P  ,     ,y, 

<  E       F{|f(x  )|  >     e  p"'n''72  } 
i  =  r+l 

=     pP{  |f(xj)|  >    £  6"V^V2}       . 

Therefore,   by  (4.15),    (4.18)  and  (4.19)  , 

P(E         ^{|S  -  S    I  >    £  n^^^}) 

*    n,  r         '    n.       ,,         r  '  -  ■" 

<  (-4^-      +  2-r  z^*^  )P(E^    ,)  +  PP{|f(x,)|  >    6  (3-V''72  } 

€"cv  '  "'"^  ^ 

This  estimate  was  obtained  under  the  assumption  that 

r  +  P  <  n.       .,,      but  ob-.'iouslv  holds  in  the  other  case  as  well. 
J, u  +  1 

Since  the    E  are  disjoint  we  have  then, 

n ,  r 

(4.20)  2      V^(E         r^(iS„.      ^,  -  S    |>     £   n^^^  }   ) 
*  ,  n,  r  ''  "J ,  u  +  1        r  '  — 

r  =1  -^ 

<      -^— +2r^P+npP{|f(x,)i  >     e^'^r}^^/!)     , 
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Now 

npP{if(xp|  >    £f3-y^^/Z  } 

^     .    2  ,2+8    ^3+8     -8/2  „-  ...     v  i  2+ S    . 
<     (-7-)  P        n  El|f(xj^)|  )     . 

Hence  as    n  ->-oo  the  second  and  third  terms  on  the  right  in  (4,  20) 
go  to  zero,   and    (3.3)    follows  immediately. 

We  have  thus  proved  the  theorem  under  the  assumption  that 
the  initial  distribution  is  the  stationary  one,    which  assunnption  we 
now  remove.     Let  TV   be  any  initial  distribution.     AA  e  show  first 
of  all  that  there  exists  a  sequence     {  P     }    of  integers  going  to 
infinity  so  slowly  that 

1/7 
(4.21)  lim      P        {      max       |S.  |  >     6n        }  =  0 

for  all     £  >  0  .     For  each    k     select  an  integer    m,      so  that 
2      1;^    {|f(x.)|  >    k-^n^/^  }<k-^ 

if    n  >    m,    .     Clearly  we  can  choose  the    m,      so  that    m,    <  m,     .   . 

And  now  let      3      =  k    if    m,    <  n  <    m,  ,,     .     Then    (3       goes  to 
n  k  -        k+1  n    '^ 

infinity  and  if    m,    <  n  <     r^i,  ,  1     then 

P{      max       |S.i>n^^^}<  Z      P      {  |f  (x. )  |  >   k"^  n^^^  } 

"^        tip„  ^  i=l      '^  ' 

<k-^  =  r^     . 

n 
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But  this  implies     (4.21)  .     It  is  obvious  that  we  can  also  choose    |3    so 
that 


(4.Z2)  p  =  o(  n^''^  )     . 


Let    p       be  the  polygon  defined  by 

p    (t)  =  S       (j-nt)  +  S.{nt-j+l),      a-l)n'^  <   t  <   jn"^     j  =  l....,n, 
n  J-i  J  ~ 

where    S..  =  0  .     And  let    p  '    be  the  polynonnial  defined  by 


p    (t)  if  0  <    t  <   p     n" 

^n  —      —  "^n 


P'{t)  = 


p    (8   n"^)  if  (3    n"^  <   t  <   1  , 

^n     n         '  '^  n         -      - 


w 


here  {p    }     satisfies     (4.21)  and  (4.22)  .     Finally,    let 


n 


p  "    =  p     -  p  '     .  Now     (4.  21)  implies  that 
'^n        ^  n         n 

(4.23)              lim  P       {        max       n"  '     p '(t)  >     €}  =  0 

for  all     £    >  0   .  Also 


P  {     max       |S.  I  >     6  n^^  }  <    (3  P{|f(xj|  >    p"^  6    n^^^  } 


,3 

e'^n 


<     — f—       E{(f(xj))^}     , 
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so  that  by     (4.22) 

(4.24)  lim       P(      max       n"^^^  p  '  (t)  >     £     }  =  0 
n-»-«K.  o<tii  ^ 

for  all     £  >  0  .     Since  we  have  shown  that  the  invariance  principle 
holds  if   TV  is  the  stationary  distribution, 

P  {  cr'^n*^^^  p     e  A  }  *-  W(A) 

1  ^n 

for  all    A  €:  ^     for  which     W(A)  =  0  .     By     (4.24)    and  Theorem  1.  4, 

(4.25)  P   {(T^^  n'^^^  p^'    e  A  }— ^W(A)  . 

-1      -1/2 
It  is  clear  that  the  set     {(T""    n'        p  "  ^  A}    is  naeasurable  on 

1  '^n 

Xfl    , ,  X      -  ,  .  .  .     sample  space.     Hence  by  Lemma  4.1, 

|P        {<r:^n~^^^  P"     €  A)   .  P  {<r-^n-^/%"    €   A}| 

<     2  r-^^— ^0  . 
By     (4.25)    then, 

^Vr  ^  ^r^  ^'^^^  P'n  ^  ^^>  ->-  W(A)    . 

But  from  this,      (4.23)    and  Theorem  1.  4    it  follows  that  if 
W (A)  =  0    then 

P      {^r^n'^^p     e  A}  ^   W(A)     . 

-rr        1  n 

We  have  thus  proved  the  invariance  principle  with  no  restrictions  on   -7T 
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§5.      The  invariance  principle  for  m~dependent  random  variables. 

A  sequence    (X    }   of    random  variables  is  said  to  be  m-depen- 

dent  if  the  random  vectors    (X_ X    ,     )  and    (X    ,     ,    ....    X    .^) 

ri'  n+r  n+s  n+t 

are  independent  whenever    s  -r  >  m.     Sequences  having  this  property 
are  of  interest  in  statistics  and  have  been  studied  by  various  authors 
(cf.   Bernstein   [2j  ,   Hoeffding  and  Robbins    [14],    Diananda    [5]    and 
Marsaglia    [l8],  )    In  this  section  we  prove  that  the  invariance  prin- 
ciple holds  for  m-dependent  sequences  if  one  or  the  other  of  two 
auxilliary  conditions  is  satisfied. 

Then  let    (Xj,)   be  an  m-dependent  sequence  of  random  vari- 
ables with  zero  means  and  finite  variances.     Let   S     =  X,+.,.+  X 

n  1  n 

and    s^    =   E{S^}  . 
n  ^    n 

Theorem  5. 1  .     If,    for  an  nn-dependent  sequence    {X^^}  ,    E{X^}  is 

bounded, 

(5.1)  |s2   -ncr^l    =  0(1) 
for  sonme  constant    <r^  >  0,    and 

(5.2)  limsn"^^"^^^       S      E{|X.|^'^M    =    0 
n-*-oo  1  =  1 

for  some    S    >  0  ,    then  the  invariance  principle  holds  for  the 

1/2 
sequence    fX^)    with  norming  factors    cTn  ' 
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Proof.      We  first  show  that  Condition    (i)   of  Theorem  3,1    is 
satisfied,   using  the  technique  of  Marsaglia    [18]    (cf.    Theorem  A.  4 
below).     Let   (n,,    .  . .  ,   n^)  be  defined  as  in   §3  ,  and  for  each  pair 
(n,  k)   with    2m  <  k  <  nc  "      and  each  j  =  1 c    define 


k-  tn 


y 


-L 


EX         ^,.    n,  J.  ,  1  <  i  <  [k"'(n.  -n.    , )] 

^,         n.    ,+  (i-l)k  +  v        •  -      -  J        J-1 


J»^  V=i  "j_l 


m 


-1, 


y.'   .  =        2  X  ^  .,          .  .            1  <  i  <  [k"(n.  -n.    ,)]     , 

J'^  v=,  n  +ik-m+v                            "J        J-1 

y!    .  =        f  X  J...         .  .            i  =  [k"Mn.-n.    ,)] 

'j.i  v=i          n  +ik-m+v                                          J       J-1 


Let 


^n,k       "  ^,,  ^j,i  •  ^n.k  u,  ^j,i 

Now  by  Holder's  inequality,   if    i  <  [k**  (n.  -n.    , )]    then 

m 

E{(y.'.)2)<m        S      E{X2  ^    }  <  m^  B   , 

'l.i         -  y_|  n.^,+ik-m+v^   -  ' 

where    B    is  the  bound  on   EfX*^}  .     Using  this  inequality,   a  similar 
one  for  the  case    i  =  [k*  (n.-n.    ,)]   and  the  fact  that  the    y!    .     are 
independent,   we  see  that 

(5.3)  E{(e^-'|^)2}  <  (n.  -n.^j)k"^  m^  B  +  (k +  m)^  B  . 

One  obtains  in  a  similar  manner  the  inequality 
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(5.4)  |E{(S     -S         )2}  -E{(gjJh2}| 

<  5(n. -n._j)k'^m2B+(k  +  m)^B  +  2m^B     . 
From    (5.3)  and    (5.4)   it  follows  that 


(5.5)  lim      lim    sup    n"     <r"     E  ((e  ^^',  )'^}  =  0 


"1     ^"2       I    rp//c  _c:  \2l     _    rp   f/^   (j)     »2 


(5.6)  lim     lim    sup    n"'   <r"^    |E{(S      -S         )'' }  -  E  {(g^^'  )^  }|    =0. 

Note  that  these  two  relations  have  been  obtained  without  the  use 
of    (5.2). 

Now  by    (5.1)  . 

(5.7)  E{(S    ..  -S    )2}    =   i<r2+  ^  , 

n+i       n 

where    3^   is  bounded.     From  this  fact  and    (5.6)  it  follows  that 

(5.8)  lim       lim    cn"^   Ct"^    ^{(gjij^  )^}    =   1  . 

where  the  iterated  linnit  is  to  be  taken  in  the  strong  sense    (cf.    the 
appendix). 

We  now  show  that  if  k   is  sufficiently  large  than  the  distri- 
bution of  the  vector 

ii^  <1)    r2    „(1)  /lite)  .-2   ^(c)    . 

^<  ^n.k  '        ^n.k <^n.k^        ^n.k  ^    ' 

where    {^       i,  )^    =    E{(g    ■'  )2}  ,    apprpaches,    as    n — *-oo,   the 
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normal  distribution  having  zero  means  and  having  the    cxc    identity 
as  its  covariance  matrix.     By  Theorenn  A.  2   it  suffices  to  show 
that  Lyapounov's  condition  is  satisfied.     But  for    k    sufficiently 
large  this  follows  from    (5,7),     (5.1)   and  the  fact  that 

But  now  from    (5.8)  and  Theorem  A.  5   it  follows  that  the  distribu- 
tion of 

_-l     -1/2   .     (1)  (c)    , 

^       "  K.k 8n.k    ^ 

approaches,    if   n ->-«   and  then  k  — >- oo  ,   the  normal  distribution 
having  zero  nneans  and  having    c"     times  the  identity  as  covariance 
matrix.     In  order  to  show  that  the  distribution  of 

<r"^n"^/^(S        S       -S^ S^     -S  ) 

n,       n-,        n,  n  n      , 

12  1  c  c-1 

approaches,   as    n-^*^c»o,   this  sanae  normal  distribution  it  is  enough, 

by  Theorem  A.  4  ,    to  show  that 

(5. 10)  p    lim      p    lim      <r"^  n"^/^   (q^^I    ,    .  . .  ,   e  ^^l   )    =    0  , 

since    S      -S  =    e      ,    +  e      ,      .     But    (5.10)   follows  immediately 

n.        n.    ,         ''n,k         n,  k  ^ 

1  1-1 

from    (5.5)  and  Chebyshev's  inequality. 

We  have  thus  proved  Condition    (i)   of  Theorem  3,1,  and  pass 
on  to  Condition    (ii).     Using  all  the  notations  of  that  theorenn. 
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P{|S    .^-S    I   >    £  n^^Vz}  <    ''s"  P{|X    I   >    6  n^^^Zm}  . 


Hence , 

(5.11)  S     P{|S   ^      -S    I   >     en^/^/2}<m       S     P{  |  X„|  >  £  n^/^/2m} 

t*=i  r+m       r     -  "  rm 

^"^<"T^  l+(S/2)         S      Eilxr'^^'^^O 

as    n — *"»"  ,    by    (5,2),     And  (making  the  inessential  assumption  that 
r  +  m<n.       ,,)  by  Chebyshev's  inequality  and    (5.7), 

P{|S  -S    .      i    >    e  n^/^Z}  <    — r-i +  — 7^ ^        . 

Hence,   since    S  -S    ,         is  independent  of    E  , 

n.       ,,        r+nn  ^  n,r 

j,u+l 


(5.12)  lim    sup        2      P(E         ^{|S  -S^      \>€n^^^/z}) 

^  n.r*^  n.       ,,        r+m '  -  /      j' 

n — >-«»       rsi  •  j.u+l 

<  V  c  V  f . 
And  now    (3.3)   follows  from    (5.11),    (5.12)  and 

(5.13)  P(E      ^^{\S  -Sj    >    en^/^   }) 

''j,u+l 

<P{|S.       -S^l    >  6  n^/^/2}  +  P(E         ^{IS  -S      .     I 

"     r+m        r'   -  '     -•         ^    n.r'^^'    n.      ^,      m+r' 

J.u+l 

>  £n^/V2}). 
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It  is  possible,   at  the  expense  of  complicating  sonaewhat  the 
proof  of  Condition  (ii)  ,   to  relax  the  condition    (5.1).     In  particular, 
it  can  be  replaced  by    s^  /^  n  O"       . 


n 


Theorem  5.  2  .     If    {X    }    is  a  stationary  nn-dependent  sequence  of 
random  variables  with  zero  nneans  and  finite  variances,    then  the 
invariance  principle  holds  for    {X    }    with  norming  factors     n        '0"  , 
where 

0-2  =  E(x2}  +  2      J^      E  {XjX^^^}     . 

Proof.     Define    y.    .,  y!    .  ,    g      ,      ,    and    e    -' ,      as  in  the  proof  of 
j,i      J, I         n,K  n,K 

Theorem    5.1.      It  is  a  simple  matter  to  show  that    (5.1)   holds  here. 

Since    EfX'^}    is  bounded  it  follows  that    (5.5)   and    (5.6)  hold  in  this 
n 

case  as  well.     In  order  to  establish  that  Condition    (i)   of  Theorem  3.1 
holds  in  the  present  case  it  suffices  to  show  that  the  vector    (5.4)   is 
asynrjptotically  normal.     But  this  follows  iminediately  by  Theorem  A,  3  . 

To  prove  the  Condition    (ii)  holds  we  proceed  as  before.     In 
fact    (5.13)  and  (5.12)  are  still  valid.     Finally, 
w 


»,?,       ^^I^r+m"^rl    >    6  n  ^  V2  }  <  mn  P{  |  Xj  >    ^n'^yinx)     , 

and  the  right-hand  side  of  this  inequality  goes  to  zero  since 
E{X2}< 


oo 
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An  immediate  consequence  of  this  theoremi  is  the  original 
result  of  Donsker    [6] « 

Theorenm  5«  3  «  If  {X^^}  is  an  independent  sequence  of  random 
variables  which  are  identically  distributed  with  zero  mean  and 
finite  variance    O"     ,   then  the  invariance  principle  holds  for 

i/z 

{ Xj^  }    with  norming  factors    n  '     CT    . 
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§6,     The  invariance  principle  for  linear  processes  with 
m-dependent  residuals. 

Let    (Y  .;   -  c^>  <  j  <  <»  }be  an  m-dependent  process  and 
^■^t'    *^-^^)    ^  sequence  of  constants  such  that  for  every  integer 
i,     2  A^  Y .    ^    converges  in  probability  to  a  variable 

t=0        t        l-t  to  f  / 

X.    =    2^    r.   -^-^  Y.    ^   as    n  -*-«>  .     Then  we  say  that 
1  t=0       t      i-t  ' 

(X. ,    -  oo  <  i  <  oo  }    is  a  discrete  linear  process  with  m-dependent 

residuals.     Processes  of  this  sort  are  of  interest  in  the  analysis 

of  time  series    (cf.    Diananda    [5]    for  references  to  the  statistical 

literature). 

Suppose  that    (Y.,-oo<  j  <  <»  }    is  a  stationary  m-dependent 

process  such  that    Y.    has  zero  nriean  and  finite  variance  and 

J 

suppose  that 


<x> 


(6.1)  2^^^   |AJ    <    oo    . 


F JT  a  fixed   i  ,    let     T^^  =  ^-n   ^t    ^ '    i      '     ^^  ^^  easy  to  show, 

using  the  m-dependence  and  stationarity  properties  of    {Y.}   , 

that 

^^<T^  +  n  -T    )2}  <  £{y2}   (    S     A2+2mA     2     |A|). 
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Therefore 

lim         sup       E{(T         -T„)2}    =    0  . 

n-^eo  U,>0  "^" 

and    {Tj^}    is  fundamental,   and  hence  convergent,   in  probair  liity. 
Thus    S     Q    ^f^-    f   converges  in  probability,   as    n-^»^««s    to  soj^ne 
random  variable    X^  =    S     _   '^^t^.t    •  ^^^    ^"^i  '     '  ^  <  ^  <  ^]   is 
a  discrete  linear  process  with  nn-dependent  residuals,     it  is 
trivial  to  show  that    {Xj}   is  stationary.     In   [5]  Diananda  has 
shown  that  the  central  limit  theorem  holds  for  processes    {Xj) 
which  arise  in  this  way,    that  is,   if    {Yj}    is  m-dependent  and  sta- 
tionary,   Yi   has  finite  variance  and    (6.1)  holds.     It  is  the  pur- 
pose of  this  section  to  prove  the  invariance  principle  for  such 
processes.     We  are  forced,  however,  to  nnake  a  stronger 
assumption  on  the  nature  of  the  sequence    {A^}  ,    viz.  ,   we 
assume  that 

(6.2)  |A„|    =    0(n"^)    . 

At  the  end  of  the  proof  we  indicate  some  ways  in  which  the  re- 
quirement  (6.2)  can  be  relaxed. 

Theorem  6. 1.     Let    {Yj,    -  oo  <  j  <  oo  }   be  a  stationary  m-depen- 
dent process  with  zero  means  and  finite  variances  and  assume 
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that    (6.2)   holds.     Then   S      „    A    Y.         converges,   as  n  — >- oo    , 

in  probability  to  some  random  variable    X-   =    2    ^   A     Y.    .     , 
^  ^  1  t  =  0       t      i-t 

so  that   X^   is  a  stationary  discrete  linear  process  with  rn-depen- 
dent  residuals,   and  the  invariance  principle  holds  for    {X^}    with 
norming  factors     n        <T"    ,     where 

(6.3)  0-2.(2    A.  )2    (E  {y2}    -i-    2     S      E{Y^Y^})    . 

Proof.     That    (X-)    exists  and  forms  a  process  of  the  type  asserted 
follows  from  the  preceeding  discussion  and  the  fact  that    (6.2) 
implies    (b.l).      We  proceed  with  the  proof  that  Condition    (i ) 
of  Theorem    3.  1   holds  ,   making  use  not  of    (6.  2)  ,    but  only  of  its 
consequence    (b.l).     Vv  e  will  use  repeatedly  the  inequality  (easily 
established  by  induction  on    n  ) 


a^  +  (aQ  +  a^)2+...  +  (a^ +  .  .  .  +  a^^^  )^ 


%?J^^^-H-'-"^Vn.l)'^   "<   if.    1-il^' 


Following  Diananda,    define 

k-1  oo 

P.    ,    =       2     A.  Y.  ,  Q,    ,    =       S     A,  Y.    , 
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^n   k    i         "  ^^ 

J-1 


P. 


i.k      • 


V      ,     .    =    n 
n.k.j 


-1/2 


n. 
J 


i=n.    ,+1 


Q 


i.k 


Now  for    k    fixed  and    i    varying,     (P.    ,  )    is  a  stationary    (m  +  k-i;- 

1  ,  K 

dependent  process  with  zero  means  and  finite  variances.     It  was 
proved  in  the  preceeding  section  that  Condition    (i)   holds  for  such 
processes    (Theorem  5.2).      Therefore  the  distribution  of 


(6.4) 


(U      ,     , U      -        ) 

n,k,l  n,k,c 


approaches,    as     n  — >-oo  ,    the  normal  distribution  having  zero 

rneans  and  having  as  covariance  matrix  the  identity  inultiplied 

2 
by     <7"      ,     where 


(6.5) 


2    _ 


„'i"L  '^("n'k.j'  =  =''<^0+---+^k-l*'   A    . 


J  '■t       »   •    *    $      ^       t 


where 


(6.6) 


tn 


X-    E  {Y^}  +  2       S      E(Y    Y^} 
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To  establish  the  second  equality  in    (6.3)   it  is  enough  to  show 
that 


(6.7)  lim     n"^   E  {(      2      P.       )2  }    =    (A^  +  • 

Let      S"  ,    P.    .     =  ^  +  >]      , 
1=1        i,k  ' 


•+\-i>'  ^ 


where 


f   =    (A„+...+A^_^)     ^S 


and 


>)=    A„Y,^  +  (A^  +  A,)Y^.,+  ...+(A„+...+A^_2)Y_^_^^2 


+  (A^+...H-A^_j)ygHA2+-..+A^_j)Y_^+...tA^_jY2_, 


Then 


E{^^}    =    (Aq+...+/.j^_j)2  (X(n-k+l)+P  )  , 
with     A    defined  by    (b.6)  and    V>     by 
(6.8)  p    =    -2        S      vE{YqY^}     . 


v=i 


If 


(6.9) 


then 


A    =    sup    I  A(.|      , 
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(6.10)  E  {  V|  2}  <  2k2    a2    E  {y2}     . 
By  Schwarz'  inequality, 

(6.11)  (E^/2{^^}-E^/^{>)2})2    <E{(?+>))2} 

<  (E^/2{52j+E^/2{>3'^))2     . 

By    (6.10)  and    (6.11)  . 

|[n-^E{(      2       R    .  )2}]^/2   -[n-N  X(n.k-H)+P   )]^/2| 

<  Zk^  A^   E  {Y^jn'^       , 

which  yields    (6.7),   and  hence    (6.5).     From  the  convergence 
of  the  distribution  of    (6.4),   and  the  limit 

?  -1        2 

lim     O-^  =  c    'c7-        , 

where    CT^    is  defined  by    (6.3),    it  follows  that  the  distribution 
of 


^"^^"n.k.l ^n.k.c^ 


approaches,    if   n-voo  and  then    k  — »•  "o      ,    the  nornnal  distribution 
with  zero  means  having  as  covariance  nnatrix   c        times  the  iden- 
tity.    In  order  to  show  that  the  distribution  of 
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^      n'(S      ,S       ~S      ,...,S       -S  ) 

^  n    '      n^        n ,  '      n  n      , 

12  1  c  c-1 

converges,   as    n — 5»-oo  ,    to  this  normal  distribution ,   it  suffices 

by  Theorem  A.  4    to  prove  that 


p    lim        p    lim       (V^^^j^^j ^n,k,c^    =    ^      ' 

or,    by  Chebyshev's  inequality,    that 

(6.12)  lim        lim    sup    E(V      ?    .)    =    0     ,  i  =  1,    .  .  .  ,    c     , 

Now,   letting    w  =  n.-n.    .   ,   one  connputes 

<^-l^)  Z2  ZZ        ^Y..t    =    ^kY,.MA,.A^^^)   Y^_^ 

i=n.    ,-f-l        t=k  ^  J 

j-1 


.  +  (A,   +...  +A,  .        ,)   Y  _^_ 

k  k-l-w-2         n.    ,+2 

+        E         (A         +•••  +A,  .     .        .)Y  ., 

k+v  k+v+w-1       n.    .  +  1-V 

v-o  j-1 


Letting 


(d.14)  B  =      max      |  E(Y^Y    }|      , 


ind  using  the  well-known  fact  that    E(z2}<lim  sup    £{2^}    if 


Tl  — >-  oo 
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Z  =  p     Urn     Zj^  ,    one  deduces  from    (6.13)   that 


n 


-1 


(6.15)  E(V^,k.j  ^-  ^-"j.p"       ^<l^kl  +  l\+li  ■^••-    )     ' 

which  yields    (6.12).     This  completes  the  proof  that  Condition  (i) 
holds. 

In  order  to  prove  that    (3,3)   holds  in  the  present  circum« 
stance,   we  first  show  that  there  exists  a  sequence    {pn)    of  integers 
tending  to  infinity  in  such  a  way  that 

(6.16)  lim     npP{|X   I    >Ep"^n^/^/2}    =    0 
and 

(6.17)  lim         S      P{|       S        Y       (A.,    +...+A  .      ., 

>    £  n^/^/4  }    =    0  . 

It  is  easy   to  show  that   X,    has  a  finite  second  nnoment.     Hence 
there  exists  an  increasing  sequence    {mj^}    such  that 


k  n  P  (IXj    >   ek"^  n^/^}  <k"^ 


if   n  >  mj^  .     If  one  puts    p^^  =  k   for    mj^  '^  "  5  "^i^  +  i  »    than    p      goes 
to  infinity  and    (6.16)   holds.     It  follows  from    (6.2)   that 
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(6.18)  Rn   =   0(n"^)  . 


where    R^  =  |A^|   +  |A^^j|   +•••     .     Now 


(6.19)  P(|      L     Y^^(A       +...+A^ 

V=0  r-  J,U+1 


)l  >    e  n'/V4} 


<  P{      S      |Y       I    R«.      >    en^/^/4} 

<    J_   P{|Y^^|    Rp^^>n^/2^p  +  v)-^/^}     . 


provided  n  is  large  enough  that 

V=o 
But 


s    (p  +  v)'^/^  <  e/4   . 


P{|Y_J   Rp^^>nl/2(p  +  v)-5A}<,-l(p^,)5/2    ^2^^ 


00 


so  that  by   (6. 18)  the  sum  in   (6.17)  is  dominated  by    S  _>.  (p  +  v) 
which  goes  to  zero  since    p   goes  to  infinity.     Hence    (6.17). 
We  now  deconnpose  the  sunmnnand  in    (3.3)   into 


+  v) 


(6.20)  P(E^^^  ^^IS^.      .^-Sj  >  £n^/^})<P{|S^^p-Sj>  £n^/V2} 

1/2 


+  P(E      ^^{|S  .S^^.bI   >    6  n^/V2})      . 
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where    (P^^    satisfies    (6.16)  and    (6.17).     It  follows  immediately 
from  stationarity  and   (6.16)  that 

M  1/7  . 

(6.21)  lim  S       P{|S    .„-S    I    >    £  n     Vz}      =      0    . 

Lie  t  n .       .  1  -  r  =  w  .     The  nS  -S      =5  +  ^»    whe  re 

l,u+i  n.       .,         r 

?  =      S      (A-^    +...  +A     .     )  Y 

v=o         P"*'V  w+v        r-v 


and 


■^=A^Y    .     +(A„+A,)Y    .       ,+...  +  (A.+  ...+A       «   , )  Y    .  „ . , 
*  0     r+w        0      1       r+w-l  0  w-p-1       r+p+1 


(9-1 


+       2      (A    +...  +A   ^       «)Y    .  . 


Hence 


(6.22)  P(E         ^{|S  -S    ..|   >    £  n^/Vz}) 

^•'^  j,u+l  ^ 

<P(E^^^)P{|>3|   >    €n^/V4}  +  P{!?   !>6n^/V4}  . 


where  the  factorization  of  the  first  term  on  the  right  is  valid  if 
n  is  large  enough  that  p  >  m  .  Now  by  stationarity  and  (6.17) 
we  have, 

(6.23)  lim      2     P  {|  ?  I    >   £  n^^/4}    =    0  . 
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Using  the  rn-dependence  property,   one  computes 

+      S      (A    +•••  +A  )2 

v=o         V  v+w-p 

00 

<   w(2A2Bm+(      2      |A^!   )^)    , 

where    A   and    B   are  defined  by    (6.9)  and    (6.14).     Since,    in  the 
notation  of  Theorem    3.1,    w  <  n/c  v»      , 

(6.24)  S      P(E         )  P{|v]   I   >   6  n^'^/4}  <  (const.  )   16   /    e^  c    v»   . 

Finally,    (3.  3)   follows  from    (6.20),    (6.21),     (6.22),     (6.23)  and 
(6.24),    connpleting  the  proof  of  the  theorem. 

It  is  clear  that    (6.2)   can  be  replaced  by  the  weaker  condition 
(6.18).     In  fact 

R„    =    0(n"^"^   )  ,  S   >  0. 

suffices.     An  examination  of  the  proof  shows  that  if  there  exists  a 
sequence     (Pf,)     going  to  infinity  in  such  a  way  that    (6.16)   and 
(6.17)   hold,    then  the  result  follows.     This  fact  can  be  used  to 
weaken    (6.2)   under  the  assumption  that   Y.   possesses  some 
moment  of  order  higher  than  two. 
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§7.     The  invariance  principle  for  recurrent  events. 

Let  (^  be  a  recurrent  event  in  the  sense  of  Feller    (cf .    [11] 
or   [12]  )  .     Suppose  that   (5   is  certain,   let 


1 '         7  '     •  •  * 

be  the  successive  recurrence  times  of   (3  and  let 


Let    Zj^   be   1   or    0   according  as    Q>  occurs  or  not  at  the  nth 
trial  and  let 


Nn  =  Zj+-«.  +Zn 

be  the  number  of  occurrences  of    0>  during  the  first   n  trials. 
Assume  that  the  recurrence  times  have  finite  nnean    M     and 
variance    <r"     .     In  this  section  we  prove  the  invariance  princi' 
pie  for  the  sequence    {Zj^  -  ii  "  }  . 


Theorem  7. 1  .     If  the  recurrence  times  have  finite  mean 


r 


and  variance    <r     ,   then  the  invariance  principle  holds  for  the 

-I  »^  /z    1/2 

sequence    {Zj^  -  ij  "    )    with  norming  factors      (T"  V>  "   '     n  ' 


-  75   - 


Proof:    Feller  has  proved  the  central  limit  theorem  for     N^ 
by  reducing  it  to  the  central  limit  theorenn  for    Sj^    via  the 
identity 

(7.1)  {N^>  k)    =    {S^<  n>. 

Our  proof  that  Condition    (i)   of  Theorem  3.1  holds  proceeds  in 

the  same  way.      Let  J  (  a, a^,)   be  the  normal  distribution 

with  zero  means  and  covariance  matrix    (^-ji).    where    X^:  =  c" 
min    (i,   j)  .     We  must  show  that 

(7.2)  lim     P   {N^  -n.  U'^  <  a.(rp"^/^n^/^.   j  =  l c} 

=     f  (aj,    ....    a^)  . 

For  j  =  1,    ....    c    let  k.  =  k.(n)  be  one  greater  than  the  integral 
part  of    n.   »-' "^  +  a.  <rw»  "^/^  n^/^  .     Then   (7.2)   reduces  to 

(7.3)  lim    P{N      <  k.  ,   j=l.    ....   c}    =    J  (a, a^)  . 

By    (7,1)   we  see  that    (7.3)   will  follow  if  we  can  prove 
fSk.-k.  1^  n-k.  p  . 


Tn-*oo 


.^-x/.^V-  tf-|) 


c 
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Since 

,  ^..-1/2    1/2 

n.  -k.  /-^  -a.CTU     '     n  ' 
J       J  3     1 

and      $(-<ii -a^,)  =  1-  5(a^,    ...,    a^ ) ,    it  suffices  to  show  that 

the  distribution  of 

--,.-1/2   1/2       ^\S     \   '\      ^ 

^ru     '     n  '  1  c 

approaches    2     ,     But  this  follows  easily  from  Theorems  A,  3  and  A,  4 

and  the  fact  that  k.  ^^^n.  U  " 

J      J  r 

We  need  a  subsidiary  result  for  the  second  part  of  the  proof. 


For  each  integer    r    let    Uj.    be  the  last  trial  before  the    (r  +  l)st 
at  which    G?    occurs,    letting    U     =  1    if  there  is  no  occurrence  of 
<^    in  the  first    r    trials.     And  let    Vj.    be  the  first  trial  after  the 
rth   at  which    <S  occurs.     Then 

(7.4)  P{V^ -U^  =  k}    =   P{Xi  =  k}  . 

For  when    (5   occurs  the  process  begins  anew    (cf,    [U])   so  that 

P{V^.U^  =  k}    =    2^,/.i,+i    P{Ur  =  v}P{Xi  =  k}  =  P{X^  =  k}     , 

To  prove  that  Condition    (ii)  of  Theorem  3.1  holds,   define 

random  variables     (3  by 

Ml ,  r        ^ 


^  11  ^ 


V  if          V     <  n.      ^, 

r  r  -    J, u+1 

"'"'           '  n.      ^,  if           V     >  n.      ^, 

J,  u+1  r         J,  u+1 


Then 

J,u+1        -^  •  » 

n,  r 

+  P(E      /^{|(N  -n.      ^,  u"^ 

^    nr '^  vi  ^    n.       ,,       j,  u+1  \       ' 
J,  u+1      -'  ' 

'  n,  r  ' 


Now    No         -N      IS    0   or    1    (accordine  as    V    >n.       , ,    or  not),    so 
Br  \  &  J.       J,  u+1  ' 

'  n,  r  ■' 

that        |(N  -p^U-').(N^-^-l)|<l+f-N(3         -r). 

n,  r  ' 


And       S  -r<V-U      ,so  that 

^n,  r  -      r  r 

P{p         -r>    f  n'/^}  <     P(X,>  |„l/2} 
''    n,  r         —    4  ^—         ^1-4  •* 

g    i/z 

Hence  if    n    is  large  enough  that   "T  n  '      >  1  t 


) 
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(7.6)  I     p{|(Np^   ^.(3^^^,.-l).(N^-r,.-^|>   |n^/^} 

<   n  P  {  Xj  >   I  n^^  }  —^  0     . 


where  the  limit  holds  because    X,    has  a  finite  second  moment. 
On  the  other  hand,   by  the  defining  properties  of  recurrent  events, 
the  second  term  of  the  second  member  of   (7.5)   is  equal  to 

P(E      )p(|N  «     -  (n.      ^,  -P)W"M  >  -T  n^/^}      . 

^^  "j,u+l"*^         '  j,u+l     ^T       '  -   2  J 

It  is  shown  in   [U,   p.  Ill]   that 

E(Nj^)  =  kj^-^  +  ^  (<r^^y^y^)y'^  -l+o(l) 


and 


Var  {Nj^l-^kiT^W  '^ 


Hence  there  exists  a  constant   A    such  that 


E{(Nj^.k|j"S^}   <   Ak 


From  this  and  Chebyshev's  inequality  it  follows  that 


J.u+l  -•  » 
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Hence 

(7.7)  hm    lim.up    Z    P(E„^„{|(N„        ^-"j,u+ir'' 

V-»»o       h  -^    -o        *^"'  J,U+1        •' 

-(Np-Pf>-^)     >   f  n^/^}  )    =    0  . 

Finally,  Condition    (ii)   follows  fro nn    (7.5),    (7,6)   and    (7.7), 
completing  the  proof  of  the  theorenn. 

It  is  possible  to  prove  this  theorenn  by  a  direct  extension  of 
Feller's  method,   avoiding  the  use  of  Theorenn  3.1  .     That  is,    it  is 
possible  to  prove  it  using  relation    (7.1)   and  the  invariance  principle 
for    {X    -  P  }     (Theorem  5.3).     While  such  a  method  is  conceptually 
appealing,    the  details  of  the  proof  become  very  involved. 
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Appendix. 

In  this  appendix  we  prove  some  multivariate  central  limit 

theorems  and  extend  to  several  dimensions  two  theorems  of 

Marsaglia   [18]  .     The  proofs  are  straightforward  but  seem  not 

to  be  available  in  the  literature  in  a  generality  sufficient  for  our 

purposes. 

Let     {V      ,   ,    k  =  1,   ,  , , ,  \>  (n),     n  =  1,  2,  , , .  }    be  an  array 
n  y  K. 

of  random  vectors 

n,k  n,  k  n,k 

We  assume  that  each    X  ^,       has  mean  zero  and  a  finite  variance 

n,k 

and  that  the  vectors  with  a  comnnon  first  subscript  are  independent. 

Let      S      ,   =    1^^"^^     X^^i,    ,      s^    .   =    E  {S^    .  }    and  let    A       be  the 
n,  j        nc=l  n,k  n,j  ^    n,j  '  n 

covariance  matrix  of  the  random  vector 

(A.  1)  (s  "  ,     S      ,,,..,   8  "         S  )  . 

n,  1       n, 1  n, c        n, c 

In  what  follows  we  assume  that 

(A.  2)  A   =    (a.    .)   =      lim    A^ 

exists.     The  matrix    A    is  of  necessity  positive  semi -definite, 
and  for  covenience  we  assume  it  to  be  positive  definite. 


-  81  - 


The  first  result  says  in  effect  that  if   (A.2)  holds  and  the 
Lindeberg  condition  holds  in  each  of  the     c     components  then  the 
central  limit  theorem  holds.     The  proof  reduces  the  problem  to 
the  central  limit  theorem  in  one  dimension,   making  use  of  a 
technique  due  to  Cramer  and  'V^old    [4]  , 

Theorem    A.  1 .       If    (A.2)  holds,    where  the  limit  nnatrix    A    is 
positive  definite,   and 

-2 


r 


t»cn) 

(A.  3)  lim       s"''.  2 


J 


(X^j|^)^        dP     =     0 

(j) 


^'     n,k  '   -   *=•     n,j' 
for  all     €  >  0    and     j  =1,    ,..,   c,    then  the  distribution  of    (A.l) 
converges  to  the  normal  distribution  having    A    as  its  covariance 
matrix  and  zero  means. 

Proof.       We  must  show  that 


C  1  c 

(A.4)  lim     E  {exp(i      S  t:  s       .    S      .)}    =    exp(-l/2       2      a.    .t.t.) 

n->oo  jsi  J     n,j      n,j  ^  ij_l       i,j    ij 

fO] 


>r  all    (t,,    ....   t^)  ,     Let  the    t.    be  arbitrary  but  fixed  throughout 
the  following  discussion.     For  notational  convenience  we  assume 


that  no    t.    is  zero.     Let 


Y      ,     =       2       t.    s   "I     X^Jj       . 
n,k  •    I        1       n,.i        n,k 


v(n) 


It  is  easily  seen  that  the  second  moment  of     2,  _  Y      ,         is 
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s^    =       S       a<"l     t.t.  . 


I     \  7 

where    a.    .     is  the     (i,  i)th  entry  of    A^  .     Note  that     s     >  0    for 
1 ,  J  '  »  ."  /  n  n 

sufficiently  large     n  ,   since     A    is  positive  definite.     If  we  can 
prove  that  the  central  limit  theorem  holds  for  the  array    {Y^   j^} 
then 

(A.  5)  lim      E   {exp{iu   s  S        Y         )}     =    e  "^   ^       . 

Since 

(A.  6)  lim      s^    =        2         a^.'^^.     t.  t.    >   0 

by    (A. 2),    substituting     (D.^.    ,    a.    .t.t,)'         for     u    in    (A.  5) 
gives    (A, 4), in  view  of  a  well-known  theorem    (cf.    Theorem  A,  5 
below) . 

In  order  to  show  that  the  central  limit  theorem  holds  for 
the  array     (  Y      ,    }    it  is  enough    (cf,  ,   e.g.  ,   [13])  to  show  that 

o       v(n)        j  ^ 

(A.  7)  lim      s'^         S  Y      ,      dP      =      0  . 

n-^oo       n         ^^^  j  n.k 

^(|Y      ,  I   >€s    } 
'      n,  k'    -         n-* 

for  all    £    >  0  .     Now  choose    S     so  that 

0  <  ^^  <  c"^    6  (       S        a.    .  t.  t.  ){    max      |t.|    )"^     , 

and  let     /»      ,      be  the  set  where     I  X      ,    I    >  "L^  s       .    for  sonne 

n,k  '     n,  k  '    -   ^      n,  J 
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j  =  1,   ....   c  .     By    (A, 6  )  and  Holder's  inequality  we  have,   for 
large    n  , 


-2 

n 


r 


X^  ,       dP       <  c  s'^       2       S 
n,k  -         n        ^^,      .,, 


^lVkl>^^n> 


'A 


g-2  (^(j)  j2    ^p 
n ,  j     n ,  k 

n,  k 


Using    (A, 6)  again,   we  see  that  in  order  to  prove     (A.  7)   it  suffices 
to  prove 


-2 


(A.  8)  lim      s  "^        S 


'^n!'k''    ^P   =    ° 


n,  k 


for     i  =1,    .  .  .  ,    c  .     But 


-2 

n,3 


'A 


(X^i>^)2    dP 


-2 


s  2 


h,k 


(X^Jh2      dP 
n,k 

"     n,k'  -  P     n,j' 


+  s 


-2 


v>Ch) 


n,k    ^'     n,k  '  ••-  •*     n,  j ' 


Now  the  first  term  of  the  right-hand  member  of  the  preceding  in- 
equality goes  to  zero  by    (A.  3),   while  the  second  term  is  dominated 
by 
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r 


< 


7       V(h) 

k=1 


dp 


c 

-2 
s 

vin) 

2 

2 

i=i 

n.i 

k  =  » 

■which  goes  to  zero,   again  by    (A.  3).     Thus  we  have  proved    (A.  8) 
and  hence  the  theorenn. 

The  multivariate  version  of  Lyapounov's  theorena  now  folJuvvs 
immediately, 

Theoremi   A.  2  .       If    (A.  2)   holds,    where  the  limit  matrix    A    is 
positive  definite,   if  the     X   -' ,       all  have  finite  moments  of  order 
2+6     for  some     o  >  0  ,     and  if 


(A.  9) 


-2-5 


lim      s 


for    j  =  l c,     then  the  distribution  of    (A.l)    converges  to  the 

normal  distribution  having    A    as  its  covarian^e  nnatrix  and  zero 
means . 

Proof.       Since 


-2 

n,  j 


(A. 3)    follows  from    (A. 9). 

A  second  consequence  of  Theorem  A.l   is  the  multivariate 
central  linnit  theorem  for  identically  distributed  random  variables. 
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Theorem.  A.  3  .       If  the    V      ,     are  identically  distributed,   each 
having  the  positive  definite  matrix    A    as  its  covariance  matrix, 
and  if      lim  v*  (n)   =  <»  ,    then  the  distribution  of    (A.  1)   converges  to 
the  normal  distribution  having    A    as  its  covariance  nnatrix  ixnd 
zero  nneans . 

Proof.       Let  the  distribution  function  of    X   •'(       be     F.  ,    where 
— r "'^  J 

x^   dF.(x)   =    (T^     .     Then 


/ 


s     .      s  {yi'-^lr  dP       =     <r  .       /         x"  dF.(x) 

n.l        ,^_,  J  n.k  J  f  jy 

^l^n^l'    ^^^n    i^  ^{|x!>£a:v^/^n)} 

n,  K  n, J  J 

goes  to  zero,   and  ihe  result  follows. 

The  next  two  theorems  are  trivial  extensions  to    c     dim^ensions 

of  results  of  Marsaglia  [18]   on  iterated  limits.     Here  an  iterated  limit 

is  used  in  the  strong  sense,   i.e.,   lim    lim   a      ,     =    a      means 

lim      lim  sup     |a-a         |    =    0.       If      V  =  (V^^^    ...,   V^^^      and 
k-».<»      n — >  do  n,  K 

V      ,     =    (V         ,    . .  .  ,   V   -'     )    are  c-dimensional  random  vectors  then  by 
n,Kn,K  n,K  ' 

p  lim      p  lim      V  =    V 

^  n ,  K 


we  mean 


lim      lim      P{|VJ^U  V^i^l    <    e,j  =   1 c}    =    1 


for  all    £   >  0  . 
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Theorem    A.  4  .       Let      V      ,     =    (V^^j   ,    ...,   ^^'^l)     and 
n,k  n,k'  n,k' 

U      1      =    (U      ,,.,,,    U      ')     be  random  vectors  such  that 
n ,  K  n ,  K  n,  K 

p  lim      p  lim      U      ,     =    0 
n,  K 

and 

lim      lim      P  { V  ^   ,    <  x.,    i  =  1,    .  .  ,  ,    c  }    -    G(x, x    ) 

v^_!^~.  n,K  ~      1      ■  L  c 

for  all  continuity  paints     (x.,    ...,  x    )    of  a  distribution  function    G  , 
Then 

lim      lim      P  (V  ^^J,  +  U^^J,  <x    .   j  =  l,    ...,   c}  =  G(x,,    ....  x^) 
k-*eo      h-*«.  ".^  n.k-     n     -^       '  r  '      c 

at  continuity  points  of     G  . 

Proof.       Marsaglia's  proof  goes  over  almost  word  for  word.     Let 
C    =    8  S  >  0      and  a  continuity  point    (x.,    .  .  .  ,   x  )  of    G    be  given. 
We  must  find  integers    K,   N, ,    N_ ,    ...       such  that 

(A. 10)  |P(Vn|l+  U^^ik5x.,   j  =  l c}    -   G(x^ x^)|     <    £ 

if    k  >  K    and    n  >  Nj^  .     First  choose     (3    so  that    (x.  -  p,    .  . .  ,   x^  -  p) 
and    (x.,  +p,   ...,   X    +p)    are  continuity  points  of    G    and  so  that 

(A.  11)  0  <G(x^  +  p x^  +  p)  .  G(x^-p x^  -p)  <    S 

Next  choose    K,   N. ,   N^ ,   ...      so  that 
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(A,  12) 
(A.13) 
(A.14) 
(A. 15) 


p(    u     (lujJh  >p})  <  S    . 

j  =  l  n,K 


|P{V^||^<x.,   ,i=l c}  -G(x^ x^)!     <     ^    . 

|P{V^J|^<x. -p.    1=1 c}  .G(xj-p.    ....  x^-|3)|    <     S  , 

|P(V^|^^<x.  +  p,    i=l,    ....   c}  -G(x^+p x^+p)!    <     ?    , 


provided    k  >  K    and    n  >  N,    ,     For  such  a  pair     (n.  k)    set 


F(§)  =  p(v^;L^u^^;)^<x.f?.j=i c). 

H(?)=    P{V„^|L+U^^;J^<x.-f?    .    |U^;»J<p.  j=l c}    . 

L(5)   c    P{V^J}^<x.  +    ?    ,    lU^Jjc'   5P'   3  =  1 ^>     • 


0(5)   =    P(\^^^l,<Xj  +   §     .    j  =  l c}     . 


T(?  )   =    G(xj  +  ?     ^c  "^^    ^ 


Then 


.|F(0)-T(0)|     <     |F(0)-H(0)|     +     |H(0)-L(0)| 


+     |L(0)   -0(0)1     +     |Q(0)  -  T(0)| 


The  first,   third  and  fourth  terms  on  the  right  are  each  less  than    8 
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by    (A.  12),     (A.  12)  and    (A.  13)   respectively.     Since 

L(  -p)<H(0)  <  L(|3) 
and 

L(-p)  <   L{0)  <    L(p)  , 
we  have 

|H(0)-L{0)|    <    |L(p)  -  L(-{3)i    <    iL{p)-0((3)| 

+   |0(P)  ~  T((3)|    +    |T(p)  -  T(-i3)|    +    |T{-(3)  -  Q(»B)| 

+    |Q(-(3)  -  L(-p)i    <     5  S 

by    {A. 12),     (A..15).    (A. 11),    (A.14)  and    (A. 12).     Hence 
|F(0)-T(0)|    <8S     ,     which  is    (A. 10). 

Theorem  A.  5  ,       l-.et     V      .      be  random  vectors  such  that 
,  ^^  j^ 

lim      Um      P{V    ^.     <  x.,   j  =  1,    ....   c}    =   G(x,,    ....   x    ) 
at  continuity  points  of    G  ,     Suppose  that 


lim      lim        A     ,    =   1 
k— *-  «o     n  —>-  oo         n ,  K 


y>\  - 

for    j  =  1,    . .  .  ,   c  .     Then 

lim      lim      P{    y^{    V^^l^  <  X      j  =  l,    ....   c}  =  G{x       ....   x^ ) 
,—^00    «—*■«>  n,K        n,K  j  1 


k— ^00 
at  continuity  points  of    G  . 
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Proof,       By  the  preceding  result  it  suffices  to  show  that 


(1) 


(1) 


(c) 


which  is  easy. 


p  am     p  lim      (  (  \l]{  -  '>^  "I <  ^n.l  -  "^n,k>  =   "  ' 
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